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Abstract 


Reparable  items  play  a  large  role  in  determining  the  readiness  of  United  States  Air  Force  (USAF) 
weapon  systems.  Many  factors  characterizing  flying  tempo  and  item  repair  and  transit  time  influence 
the  level  of  fleet  readiness.  Readiness-Based  Leveling  (RBL)  considers  these  factors  as  it  seeks  to 
maximize  aircraft  readiness  as  it  allocates  spare  reparable  items  between  bases  and  a  depot.  The  purpose 
of  this  research  was  to  demonstrate  the  validity  of  using  response  surface  methodology  (RSM)  within  the 
context  of  RBL  in  an  effort  to  quantify  the  influences  these  factors  have  on  aircraft  readiness. 

RSM  applied  designed  experiments  and  least  squares  regressions  in  developing  a  series  of 
empirical  models  quantifying  correlations  between  one  uncontrollable  and  seven  controllable  factors  and 
RBL’s  output.  Verification  tests  indicated  the  empirical  models  represented  -  to  a  high  degree  --  the 
quantitative  relationships  present  between  the  inputs  and  output  of  RBL.  Although  valid  conclusions 
cannot  be  made  from  the  models  (a  substitute  input  was  used  in  place  of  a  usual  D041  input),  the 
methodology  as  demonstrated  is  valid. 


APPLYING  RESPONSE  SURFACE  METHODOLOGY 


TO  READINESS-BASED  LEVELING  OF  REPARABLE  ITEMS 

I.  Introduction 

America  is  committed  to  defending  its  interests  at  home  and  abroad.  The  United  States  Air 
Force  operates  many  kinds  of  aircraft  as  one  component  of  America’s  defense  capability.  Today’s  aircraft 
are  veiy  complex,  technological  marvels  with  high  price  tags  and  sometimes  may  not  be  available  for 
missions.  What  keeps  an  aircraft  from  being  available?  First,  aircraft  items  alone  can  be  a  source  of 
aircraft  nonavailability  when  required  replacements  are  not  available.  Second,  if  a  replacement  is 
available,  the  maintenance  required  to  install  or  test  the  item  may  keep  an  aircraft  nonavailable.  This 
research  is  concerned  with  the  first  cause  of  aircraft  nonavailability. 

Each  aircraft  item  is  one  of  two  types:  reparable  and  nonreparable.  A  reparable  item  is  capable 
of  being  repaired,  and  a  nonreparable  item  is  not  capable  of  being  repaired.  A  line-replaceable  unit  (LRU) 
is  an  example  of  a  reparable  item.  Oil  is  an  example  of  a  nonreparable  item.  When  comparing  these  two 
types  of  items,  reparable  items  have  more  impact  on  an  aircraft’s  availability  to  fly  (Sherbrooke,  1992:5). 
Reparable  items  tend  to  be  more  expensive  and  account  for  the  largest  portion  of  the  spares  budget:  $3 1 
billion  of  spare  reparable  items  in  the  Air  Force’s  1990  budget  (Sherbrooke,  1992:5).  High  costs  are 
apparent  in  the  number  of  reparable  items  on  an  aircraft:  F-1 1 1  with  3758,  F-16  with  3306,  B-1  with  2650 
(Rexroad,  1992:  24).  Although  reparable  items  have  relatively  low  demand  at  the  base  level,  they  have 
longer  lead  times  than  nonreparable  items.  A  long  lead  time  means  it  would  take  a  long  period  of  time  to 
solve  a  supply  problem  (Sherbrooke,  1992:5,6). 

Managing  a  reparable  item  involves  large  quantities,  high  cost,  low  demand,  long  lead  time,  and 
impact  on  aircraft  availability.  Managing  a  set  of  reparable  items  is  a  complex  and  challenging  process. 
Two  of  the  more  significant  challenges  involve  determining  the  quantity  of  each  item  needed  and  to  whom 
should  they  be  allocated.  This  research  is  concerned  with  the  second  challenge,  also  known  as  central 
stock  leveling  (CSL). 
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Mathematical  models  are  available  for  CSL.  In  both  the  actual  CSL  system  and  its  model,  certain 
system  characteristics  may  have  an  influence  on  the  final  output  or,  in  this  case,  how  reparable  items  are 
allocated.  In  other  words,  there  may  be  a  relationship  between  the  model’s  inputs  and  output(s). 

Learning  about  such  relationships  in  a  system’s  model,  if  they  exist,  can  increase  general  knowledge  about 
the  system’s  model,  if  not  the  actual  system  under  study. 

Research  Objective 

The  objective  of  this  research  was  to  develop  a  method  for  learning  about  possible  relationships 
between  the  inputs  and  output  of  Readiness-Based  Leveling  (RBL).  RBL  performs  CSL  via  the  Multi- 
Echelon  Technique  for  Recoverable  Item  Repair  (METRIC)  model.  RBL  and  METRIC  are  discussed  in 
more  detail  in  Chapter  11. 

RBL  is  designed  to  allocate  the  items  in  a  fashion  that  minimizes  the  number  of  expected  base 
backorders  for  the  item.  Minimizing  backorders  is  important  because  it  is  equivalent  to  maximizing 
aircraft  availability  (Sherbrooke,  1992:38,39).  A  lower  number  of  expected  backorders  translates  to  a 
higher  availability  of  aircraft  for  conducting  military  operations. 

Air  Force  Material  Command  (AFMC)  Studies  and  Analysis  (SAO)  sponsored  this  research  to 
investigate  which  factors  or  variables  affecting  RBL  are  most  important.  This  research  developed  an 
empirical  model  of  RBL  to  determine  if  every  aspect  of  RBL  is  equally  important  or  influential  in 
determining  the  number  of  expected  base  backorders. 

Scope 

This  research  focused  on  the  aspect  of  the  supply  system  dealing  with  allocating  reparable  items 
between  the  bases  and  the  depot.  Therefore,  it  did  not  focus  on  the  process  for  determining  how  many 
reparable  items  to  allocate.  Furthermore,  the  research  only  considered  two  echelons  -  base  and  depot  -- 
and  a  first-indentured  item.  One  item  and  one  aircraft  system  (F-16)  were  considered. 

RBL  was  not  critiqued  in  this  study.  It  is  one  of  several  techniques  for  CSL.  Other  techniques 
include  the  Aircraft  Availability  Model  (AAM),  Distribution  and  Repair  in  Variable  Environments 
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(DRIVE)  model,  and  fixed  safely  levels  (FSL).  An  evaluation  of  the  four  techniques  mentioned  here, 
including  METRIC,  was  performed  as  part  of  RAND’s  Project  AIR  FORCE  (Miller,  1995), 

Summary 

This  chapter  described  the  complex  nature  and  large  budget  expense  surrounding  Air  Force 
reparable  items.  Management  has  several  CSL  alternatives  for  allocating  reparable  items  between  the 
bases  and  the  depot.  AFMC  has  supported  the  use  of  RBL  with  its  incorporated  METRIC  model  to  solve 
the  allocation  problem.  In  response  to  inquiries  from  HQ  AFMC/S  AO,  this  research  developed  an 
empirical  model  of  RBL  to  determine  which  aspects  of  the  multi-echelon,  reparable  item  environment 
most  influence  the  expected  base  backorders  for  a  reparable  item. 

Overview 

Chapter  II  presents  background  information  relevant  to  this  research.  It  discusses  RBL  in  more 
detail  and  introduces  an  empirical  modeling  technique  referred  to  as  Response  Surface  Methodology 
(RSM).  Chapter  III  explains  the  methodology  of  this  research.  It  discusses  factors  to  be  considered,  some 
preliminaiy  calculations,  and  the  experimentation  process.  Chapter  IV  discusses  the  results  of  the 
experiment  and  resulting  empirical  model,  while  Chapter  V  closes  the  paper  with  conclusions  and 
recommendations.  For  easy  reference  to  most  supply-related  terms  and  the  usual  large  population  of  Air 
Force  acronyms,  the  reader  is  referred  to  the  List  of  Terms  included  in  this  document. 
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li.  Literature  Review 


Introduction 

Response  Surface  Methodology  (RSM)  was  applied  to  develop  an  empirical  model  of  Readiness- 
Based  Leveling  (RBL).  RBL  maximizes  aircraft  readiness  as  spare  parts  are  allocated  between  the  bases 
and  the  depot.  After  presenting  the  general  purpose  and  process  of  RBL,  there  is  an  exposition  of  its 
underlying  model:  Multi-Echelon  Technique  for  Recoverable  Item  Control  (METRIC),  METRIC 
considers  two  supply  echelons,  base  and  depot,  as  it  utilizes  marginal  analysis  to  distribute  spare  parts  and 
figure  the  number  of  expected  backorders  resulting  from  various  distribution  schemes.  A  brief  section  on 
multi-echelon  theory  concludes  with  the  computational  process  of  METRIC. 

The  RSM  section  incorporates  discussion  of  its  purpose  and  fimctionalily.  A  section  on  empirical 
models  discusses  how  the  true  output  response  surface  must  be  approximated  because  of  unknown 
relationships  between  the  process  output  response  and  input  variables.  Experimental  design  is  an  integral 
part  of  RSM,  and  so,  along  with  desirable  design  characteristics,  it  is  discussed.  There  is  a  section  on  two 
types  of  error  to  be  aware  of  in  RSM:  random  and  bias.  A  section  on  canonical  analysis  and  its  role  in 
analyzing  second-order  polynomials,  along  with  a  chapter  summary,  complete  this  chapter. 

Readiness-Based  Leveling  (RBL) 

This  section  provides  a  background  on  the  purpose  and  process  of  RBL.  Although  Readiness- 
Based  Leveling  was  coined  by  AFMC/SAO  during  conference  discussions  in  1995,  its  roots  date  back  to 
the  1960s  when  its  underlying  solution  procedure,  the  Multi-Echelon  Technique  for  Recoverable  Item 
Control  (METRIC),  was  developed  (Sherbrooke,  1966).  An  exposition  of  METRIC  follows  a  discussion  of 
the  purpose  of  RBL. 

RBL  Purpose 

The  purpose  of  Readiness-Based  Leveling  is  in  its  name.  Leveling  is  the  activity  of  allocating 
spares  of  reparable  items  (i.e.  stock)  between  the  bases  and  the  depot.  Going  one  step  flirther,  RBL  is  a 
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central  stock  leveling  (CSL)  method  because  the  allocation  determination  is  made  at  a  centralized 
location,  Readiness-Based  means  the  allocation  is  done  in  a  manner  that  maximizes  Air  Force  readiness. 
In  this  context,  the  measure  of  readiness  is  aircraft  availability.  RBL  minimizes  expected  base  backorders 
for  a  reparable  item.  As  stated  in  Chapter  I,  minimizing  backorders  is  important  because  it  is  equivalent 
to  maximizing  aircraft  availability  (Sherbrooke,  1992:38,39), 


Multi-Echelon  Technique  for  Recoverable  Item  Control  (METRIC) 

METRIC  provides  the  procedure  for  meeting  the  goal  of  RBL:  allocation  of  reparable  (that  is, 

recoverable)  items  in  a  fashion  that  maximizes  readiness  by  minimizing  expected  backorders.  The 

model’s  developer  describes  it  as  follows: 

METRIC  is  a  mathematical  model  translated  into  a  computer 
program,  capable  of  determining  base  and  depot  stock  levels  for  a 
group  of  recoverable  items;  its  governing  purpose  is  to  optimize 
system  performance  for  specified  levels  of  system  investment. 

METRIC  is  designed  for  application  at  the  weapon-system  level, 
where  a  particular  line  item  may  be  demanded  at  several  bases  and  the 
bases  are  supported  by  one  central  depot  (Sherbrooke,  1966:2). 


METRIC’S  Purposes 

Three  purposes  of  the  METRIC  model  are 

1.  Optimization.  A  major  purpose  of  the  model  is  to  determine  optimal 
base  and  depot  stock  levels  for  each  item,  subject  to  a  constraint  on 
system  investment  or  system  performance... 

2.  Redistribution.  The  model  can  take  fixed  stock  levels  on  each  item 
and  optimally  allocate  the  stock  between  the  bases  and  depot.... 

3.  Evaluation.  The  model  provides  an  assessment  of  the  performance 
and  investment  cost  for  the  system  of  any  allocation  of  stock  between 
the  base  and  depot  (Sherbrooke,  1966:2). 


metric’s  Mathematical  Assumptions 

L  “System  Objective  of  Minimizing  the  Expected  Number  of  Backorders”  (Sherbrooke,  1966:6). 

The  objective  is  defined  in  this  way  :  “Take  a  fixed  period  of  time  and  add  together  the  number  of  days  on 
which  any  unit  of  any  item  at  any  base  is  backordered.  Dividing  this  number  by  the  length  of  the  period 
and  taking  the  expected  value  of  the  statistic  yields  a  number  that  is  independent  of  the  period  length 
(Sherbrooke  1966:  6).  This  objective  was  preferred  because  “...backorders  are  a  convex  function  of  base 
stock  level  when  the  depot  stock  level  is  constant”  (Sherbrooke  1966:  6),  Sherbrooke  showed 
impracticalities  of  using  other  objective  functions  which  were  maximizes:  fill  rate,  ready  rate,  operational 
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rate.  This  assumption  is  reasonable  for  a  goal  of  maximizing  aircraft  availability  because  both 
minimizing  backorders  and  maximizing  availability  were  stated  to  be  equivalent. 

2.  “Compound  Poisson  Demand”  (Sherbrooke  1966:  8). 

This  assumption  has  not  followed  observation  (Sherbrooke,  1992:47).  First  of  all,  there  are  no 
observations  of  clusters  of  demand  (Sherbrooke,  1992:62).  Second  of  all,  a  compound  Poisson 
distribution  of  demand  is  characterized  by  a  variance  to  mean  ratio  (VTMR)  that  is  constant  over  the 
demand’s  observed  time  period.  However,  empirical  data  has  shown  the  VTMR  to  increase  with  time 
(Sherbrooke,  1992:61,62).  Hence,  the  negative  binomial  distribution  is  used  to  generalize  a  Poisson 
distribution  (with  nonconstant  mean)  while  allowing  a  dynamic  VTMR.  The  two  parameters  used  to 
define  the  negative  binomial  distribution  are  an  item’s  mean  demand  rate  and  VTMR  (Sherbrooke, 
1992:61). 

3.  “Demand  is  Stationary  Over  the  Prediction  Period”  (Sherbrooke  1966:  9). 

For  time  intervals  of  equal  length  within  the  prediction  period,  it  seems  reasonable  that  the  distribution  of 
demands  for  an  item  will  be  the  same.  This  assumption  would  appear  to  be  more  true  for  prediction 
periods  with  a  consistent  environment,  that  is  at  peace  or  at  war. 

4.  “Decision  on  Where  Repair  Is  to  Be  Accomplished  Depends  on  the  Complexity  of  the  Repair  Only” 
(Sherbrooke  1966:  10). 

This  assumption  also  means  base  workload  has  no  impact  on  where  the  item  will  be  repaired.  A  study 
found  .3  percent  of  10,965  items  were  sent  to  the  depot  for  workload  reasons  (Weifenbach,  1966:  4). 

5.  “Lateral  Resupply  IBetween  Basesl  is  Ignored  “(Sherbrooke  1966:  10). 

This  assumption  is  reasonable  since  the  number  of  lateral  shipments  “is  typically  small” 

(Sherbrooke  1966:  10). 

6.  “System  is  Conservative”  [Items  Are  Not  Condemned]  (Sherbrooke  1966:  10). 

A  study  found  only  4.1  percent  of  10,965  items  were  condemned  (Weifenbach,  1966:  4).  “The 
condemnation  rate  must  be  considered  for  procurement  purposes,  but  the  procurement  process  in  not 
considered  in  the  METRIC  optimization”  (Sherbrooke,  1966: 10-1 1). 

7.  “The  Depot  Does  Not  Batch  Units  of  a  Recoverable  Item  for  Repair  Unless  There  is  an  Ample  Supply 
of  Serviceable  Assets”  (Sherbrooke  1966:  11). 

“In  those  few  cases  where  setup  cost  is  an  important  factor  and  demand  is  reasonably  high,  so  that  some 
batching  is  indicated,  the  estimate  of  depot  repair  time  should  include  the  average  waiting  time  before 
depot  repair  is  initiated”  (Sherbrooke,  1966:11).  The  METRIC  model  used  in  this  research  factored  in 
depot  delay. 

8.  “Recoverable  Items  May  Have  Different  Essentialities”  (Sherbrooke  1966:  11). 

It  seems  reasonable  to  assume  different  types  of  items  may  have  associated  with  them  degrees  of  need  by 
the  bases.  Some  items  may  restrict  the  capability  of  an  aircraft  more  than  another. 

9.  “Demand  Data  from  Different  Bases  Can  Be  Pooled”  (Sherbrooke,  1966:12). 

Any  averaging  technique  deemed  satisfactory  by  the  user  may  be  used  to  come  up  with  an  initial  estimate 
of  demand  per  flying-program  element:  per  flying  hour,  or  for  this  research,  per  100  flying  hours. 
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Multi-Echelon  Theory 


Bases  and  the  depot  make  up  the  multi-echelon  environment.  Certain  types  of  characteristics  at 
both  locations  are  of  interest.  One  characteristic  is  the  demand  rate  for  an  item  at  the  depot,  which  is  a 
function  of  the  demand  rate  for  an  item  at  the  base. 

The  mean  depot  demand  rate  is 

y  y  (2-1 

u  (/V  ijJ 

y=i  y=i 


where 

X'lj :  mean  customer  arrival  rate  with  item  i  at  base  j 

fij  :  mean  demand  per  customer  with  item  i  at  base  j 

rij  :  probability  item  i  can  be  repaired  at  base  j 

(1-ry) :  probability  item  i  at  base  j  must  be  shipped  to  depot  for  repair 

0ij :  mean  demand  for  item  i  at  base  j  (Sherbrooke,  1966: 13). 

The  expected  backorders  at  a  point  in  time  for  a  given  stock  level  is 

oo 

B(s)=  '^{x-s)p{x\AT,VJMR)  (2-2) 

x=s-^l 


where 

s  :  spare  stock  for  an  item  defined  as  the  sum  of  stock  on  hand  plus  on  order  plus  in 
repair  minus  backorders;  a  constant  value  under  the  one-for-one  replacement  policy 
X  :  mean  customer  arrival  rate  where  demand  for  an  item  is  negative  binomial* 

T  :  mean  resupply  (repair)  time  for  an  arbitrary  distribution  T(t) 

p(x|?iT,  VTMR) :  negative  binomial*  probability  density  for  a  mean  customer  rate  XT 

(Sherbrooke,  1966: 13, 14) 

*  Recall  the  discussion  in  METRIC’s  second  mathematical  assumption. 


METRIC’S  Computational  Process 

There  are  five  stages  to  computing  the  solution.  Only  one  type  of  item  is  considered  through  the 
first  four  stages.  Stage  five  considers  all  types  of  items  simultaneously  and  is  included  for  completeness. 
The  scope  of  this  research  considers  only  one  item.  “The  essential  idea  is  to  compute  the  depot  delay  in 
resupplying  bases,  and  this  depends  on  the  depot  stock  level.  Then  we  can  compute  the  backorders  at 
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each  base  which  depend  on  the  resupply  delay  from  depot  and  the  base  stock  level”  (Sherbrooke,  1992:48). 
The  five  stages  are  briefly  explained. 

Stage  1:  Compute  depot’s  average  delay  per  demand 

The  depot’s  average  response  time  to  a  base’s  resupply  request  is  a  fimction  of  the  depot  spare 
stock,  Sio.  The  bounds  on  this  response  time  can  be  seen  when  the  depot  has  either  an  infinite  or  zero 
amount  of  spare  stock.  If  the  spare  stock  is  infinite,  then  the  response  time  is  just  the  average  order  and 
ship  time,  Oij.  If  the  spare  stock  is  zero,  then  the  depot’s  average  repair  time,  Dj,  must  be  added  to  the 

average  order  and  ship  time.  Thus,  the  delay  at  the  depot  will  be  between  zero  and  the  depot’s  average 
repair  time  (Sherbrooke,  1966: 14). 

The  depot  delay  can  be  computed  as  a  fimction  of  the  depot  spare  stock.  Let  %{ be  the  expected 
number  of  customers  for  item  i  who  arrive  at  the  depot  in  a  certain  time  period;  so  X\  =  EX.ij(l-rij)  for  all 
bases  j.  A  resupply  delay  will  occur  at  the  depot  when  there  are  more  items  in  repair  than  there  were  in 
the  spare  stock.  In  the  case  of  a  resupply  delay,  the  difference  between  the  items  in  repair  and  the  spare 
stock  quantity  denotes  how  many  items  are  being  delayed.  Using  Equation  2-2,  the  expected  quantity 
being  delayed  at  any  point  in  time  is  (Sherbrooke,  1966: 14): 
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The  total  expected  system  delay  over  any  time  period  is  simply  the 
expected  number  of  units  on  which  delay  is  being  incurred  at  a 
random  point  in  time  multiplied  by  the  length  of  the  time  period. 
Since  we  are  interested  in  the  average  delay  per  demand,  we  must  then 
divide  by  the  expected  number  of  demands  over  that  time  period.... 
Thus,  the  average  delay  per  demand.. .is  (Sherbrooke,  1966:15): 
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Stage  2:  Compute  Expected  Backorders 

Expected  backorders  for  each  level  of  depot  stock,  Sio,  is  a  function  of  the  base  stock,  Sij. 


Equation  2-2  is  used  assuming  that  s=Sij,  A,=Xij,  and  T  =  ryAij  +  (l-rij)(Oj  +  5(Sio)Di)  (Sherbrooke, 
1966:15). 

Stage  3:  Determine  Ontimal  Allocation  of  Items  to  the  Bases 

For  each  level  of  depot  stock,  Sio,  the  marginal  allocation  procedure  determines  where  to 

distribute  the  next  unit  of  item  i.  The  criteria  used  is  which  base  would  experience  the  largest  drop  in 
backorders  as  a  consequence  of  receiving  the  additional  unit  (Sherbrooke,  1966:15). 

Stage  4:  Select  Minimum  Expected  System  Backorders 

The  minimum  expected  system  backorders  is  selected  for  each  level  of  constant  total  system 
stock,  Sio+Sij.  For  example,  a  total  system  stock  of  ten  units  of  item  i  could  be  distributed  between  the 
depot  and  bases  in  various  ways:  one  at  depot  and  nine  to  the  bases,  two  at  de{>ot  and  eight  to  the  bases, 
and  so  on.  Associated  with  each  distribution  scheme  for  this  constant  stock  level  is  an  expected  number  of 
backorders  at  the  bases.  This  stage  will  select  for  one  type  of  item  —  independently  of  the  other  types  of 
items  —  that  distribution  scheme  corresponding  to  the  least  expected  backorders.  (Sherbrooke,  1966:16). 

Stage  5:  Determine  Next  Investment  Among  the  Items 

Now  we  consider  the  multi-item  problem.  Marginal  analysis  is  again 
employed.  Using  the  item  backorder  functions  computed  in  Stage  4, 
the  next  investment  is  allocated  to  that  item  which  produces  the 
maximum  decrease  in  expected  backorders  divided  by  unit  cost.  This 
[marginal  analysis  approach]  is  similar  to  the  procedure  we  used  in 
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Stage  3,  except  that  there  we  were  dealing  with  a  single  item  so  that 
unit  cost  was  not  a  variable....  After  each  allocation,  the  system 
investment  and  system  backorders  are  computed.  Allocation 
terminates  when  the  investment  target  is  just  exceeded  or  alternatively, 
when  the  expected  backorders  are  just  less  than  a  target  value 
(Sherbrooke,  1966:  16). 

Since  only  one  item  was  analyzed  in  this  research.  Stage  5  was  not  included  in  the  analysis  but  is  shown 
here  for  completion. 


Summary 

RBL  considers  different  aspects  of  the  multi-echelon  environment  as  it  takes  a  quantity  of 
reparable  items,  one  type  at  a  time,  and  distributes  them  between  the  bases  and  depot.  RBL  uses  marginal 
analysis  to  determine  where  the  next  unit  should  be  distributed.  It  minimizes  expected  base  backorders 
which  is  equivalent  to  maximizing  aircraft  availability.  Perhaps  not  all  aspects  are  equally  important  in 
affecting  RBL’s  outcome.  RSM  is  a  mathematical  tool  which  can  be  used  to  determine  the  degree  of 
influence  factors  have  on  a  particular  ^stem. 

Response  Surface  Methodology  (RSM) 

This  research  applied  RSM  to  develop  an  empirical  model,  or  function,  for  RBL. 

Response  surface  methodology  comprises  a  group  of  statistical 
techniques  for  empirical  model  building  and  model  exploitation.  By 
careful  design  and  analysis  of  experiments,  it  seeks  to  relate  a 
response,  or  output  variable  to  the  levels  of  a  number  of  predictors,  or 
input  variables,  that  affect  it  (Box  and  Draper,  1987:1). 

This  brief  discussion  of  RSM  includes  a  description  of  the  structure  of  an  empirical  model,  as 
applied  to  RSM.  Then,  experimental  design,  which  provides  the  structure  for  data  generation  to  assist 
model  building,  is  covered.  Models  fall  short  of  representing  their  respective  process  perfectly,  implying 
the  presence  of  error;  two  types  of  error  can  exist  in  RSM,  random  and  bias.  A  discussion  of  these  errors 
is  followed  by  a  description  of  canonical  analysis,  useful  in  providing  a  straightforward  description  of  the 
approximated  response  surface.  Three  experimental  design  characteristics  helpful  in  minimizing  random 
and  bias  error  will  complete  this  chapter. 
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Building  An  Empirical  Model 


Process  experts  understand  how  a  process  works;  they  know  what  makes  it  run  well  or  not  run 
well.  If  something  related  to  the  process  changes,  they  know  how  it  will  affect  the  process  because  they 
understand  underlying  relationships  governing  it.  This  knowledge  is  acquired  over  time  and  may  not  be 
easy  to  convey  to  someone  inexperienced  with  the  process. 

RSM  is  a  technique  that  enables  a  person  to  become  an  “expert”  about  a  process,  so  to  speak. 
Before  applying  RSM  to  a  process,  an  analyst  may  not  know  anything  about  the  process.  He  or  she  may 
not  know  which  factors  are  meaningful  or  what,  if  any,  relationships  exist  among  the  factors  or  between 
the  factors  and  the  process  output.  However,  with  some  time-saving  advice  from  the  experts  at  the 
beginning  of  an  analysis,  RSM  can  shed  light  on  process  characteristics  that  were  not  only  unknown  to  the 
analyst,  but  also  may  not  have  been  totally  understood  by  the  experts.  The  time-saving  expert  advice 
includes  which  factors  are  likely  to  be  meaningful  to  the  process,  what  is  the  process  output,  and  where 
can  data  be  collected. 

The  product  of  incorporating  expert  advice  and  RSM  is  an  empirical  model  of  the  process.  A 
graphical  depiction  of  the  relationship  between  the  factors  and  the  process  output  is  called  a  response 
surface.  A  true  response  surface  represents  the  true  process  outcome.  The  empirical  model  represents  the 
response  surface  in  equation  form.  There  can  be  many  possible  models,  all  representing  the  process  and 
its  true  response  surface  to  different  degrees  of  accuracy.  Two  examples  of  models  representing  a  process 
where  at  least  three  factors  were  considered  are: 

gi  (x,P)  =  Po  +  Pi  Xi  +  p2  X2  +  P3  X3  (2-6) 

g2  (X,P)  =  Po  +  Pi  Xi  +  p2  X2  +  p3  X3  +  Pl3  X]  X3  (2-7) 

I  say  “at  least  three  factors  were  considered”  because  more  factors  may  have  been  considered  originally 
but  were  determined  to  not  be  meaningful  in  representing  the  process.  The  factors  are  represented  by  the 
vector  X.  The  vector  p  represents  how  the  factors  influence  the  process  output.  Model  g2  (x,P)  contains 
one  more  term  than  gi  (x,P).  Since  the  models  differ,  they  will  have  different  accuracies  in  representing 
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the  process  output.  Perhaps  the  extra  term  in  g2  (x,P)  makes  a  positive  difference,  enhancing  the  model’s 
accuracy. 

An  RSM  empirical  model  “can  be  thought  of  as  a  Taylor’s  series  expansion  of  the  true 
underlying  theoretical  function.. .truncated  after  terms  of  dth  order”  (Box  and  Draper,  1987:21).  Higher 
degree  polynomials  usually  provide  a  better  approximation  of  the  true  response  surface.  For  a  polynomial 
of  degree  d,  a  smaller  region  of  interest  usually  results  in  a  better  approximation.  “Region  of  interest” 
refers  to  the  range  of  values  over  which  the  factors,  or  input  variables,  can  vary.  In  practice,  a  first  or 
second  degree  polynomial  might  be  adequate  for  a  limited  range  of  the  factors  (Box  and 
Draper,1987:21,22).  Equations  2-6  and  2-7  above  represent  possible  first  and  second  degree  polynomials, 
respectively.  A  more  formal  representation  of  a  second  degree  polynomial  approximation  of  the  response, 
y,  has  the  following  form: 

>)  =  K+tb.x,  +1:  (2-8) 

1=1  /=!  7=1  >=i+l 

The  statistical  techniques  used  in  developing  RSM’s  empirical  models  include  experimental 
design,  least  squares  regression,  and,  if  the  result  is  a  second  order  polynomial,  canonical  analysis.  The 
following  sections  briefly  describe  these  techniques,  with  the  exception  of  least-squares  regression;  the 
reader  is  assumed  to  be  familiar  with  least  squares  regression. 

Experimental  Design 

The  source  of  the  data  used  to  develop  an  empirical  model  can  be  either  prior  observation  or 
planned  experimentation.  In  the  context  of  RSM,  designed  experiments  provide  virtually  all  of  the  data 
for  empirical  model  building  and  exploitation.  A  designed  experiment  is  “a  test  or  series  of  tests  in  which 
purposeful  changes  are  made  to  the  input  variables  of  a  process  or  system  so  that  we  may  observe  and 
identify  the  reasons  for  changes  in  the  output  response”  (Montgomeiy,  1991: 1).  Many  experimental 
designs  are  efficient  in  that  they  provide  a  great  deal  of  information  from  relatively  few  experiments,  or 
design  points. 
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An  experimental  strategy  can  involve  a  series  of  tests.  A  screening  design  might  constitute  the 
first  experiment.  A  screening  design  is  a  relatively  small  scale  (in  terms  of  the  data  generated) 
experiment  that  assesses  the  significance  of  all  of  the  original  factors.  Those  factors  exhibiting  only  a 
negligible  influence  on  the  process  output  can  be  eliminated  from  fiirther  consideration  and 
experimentation.  Such  screening  reduces  the  dimensionality  of  all  the  subsequent  designs  included  in  the 
sequential  experimental  strategy. 

“Purposeful  changes”  to  the  values,  or  levels,  of  the  factors  can  be  implemented  simultaneously 
or  by  one-at-a-time.  During  one-at-a-time  changes,  one  factor  level  is  varied  as  all  other  factor  levels 
remain  constant.  Purposeful,  simultaneous  changes  to  the  factor  levels,  done  in  context  of  an 
experimental  design,  are  more  efficient  in  gathering  meaningful  data  and  can  also  detect  interactions 
among  the  factors;  one-at-a-time  changes  cannot  detect  interactions. 

As  a  result  of  purposeful,  simultaneous  changes  to  the  factor  levels,  the  analysis  to  determine 
reasons  for  changes  in  the  output  response  is  relatively  straightforward.  The  “reasons”  most  likely  lie  in 
the  correlations  that  may  exist  between  the  output  and  the  factors.  The  resulting  form  of  the  empirical 
model  will  approximate  these  relationships. 

Desirable  Design  Characteristics 

There  are  certain  characteristics  desirable  in  experimental  designs:  orthogonality,  rotatability, 
and  small  number  of  design  points  to  name  a  few.  A  design  is  orthogonal  if  all  “n-dimensional  design 
vectors... are  at  right  angles,”  implying  the  factors  are  uncorrelated  with  each  other  (n  denotes  number  of 
observations.  There  is  an  n-dimensional  design  vector  for  each  factor  included  in  the  experiment)  (Box 
and  Draper,  1987:479).  Orthogonality  is  desirable  because  it  allows  all  the  factor  effects  to  be  measured 
independently  from  each  other  (Schmidt  and  Launsby,  1989:3.2).  Not  all  designs,  however,  are 
orthogonal.  The  demands  of  orthogonality  often  require  more  runs  than  is  practical  in  an  experimental 
setting.  Fortunately,  orthogonality  is  not  critical;  modem  computing  technologies  allow  estimation  of 
factor  effects  even  when  design  are  not  orthogonal  (Box  and  Draper,  1987:510).  In  order  to  simplify  the 
process  of  developing  orthogonal  designs,  factor  levels  are  usually  coded  in  the  following  way: 
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(2-9) 


where  is  the  mean  of  the  settings  of  factor  i 

Si  is  the  half-width  of  the  range  of  settings  for  factor  i 

The  result  of  such  a  coding  scheme  is  the  highest  setting  for  each  factor  takes  on  the  value  of  one  while 
the  lowest  setting  takes  on  a  value  of  minus  one  (Box  and  Draper,  1987:480). 

An  experimental  design  is  rotatable  if  the  variance  of  the  empirical  model’s  predicted  response 
is  a  function  of  a  design  point’s  distance  from  the  design  center  (Box  and  Draper,  1987:440).  All  design 
points  equidistant  from  the  design  center  are  equally  important  in  terms  of  the  information  they  contain 
about  the  response  surface.  The  variance  associated  with  the  information  is  the  same  for  each  of  the 
equidistant  points.  Therefore,  a  rotatable  design  provides  the  same  amount  of  accuracy  in  every  direction 
from  the  design  center.  In  other  words,  none  of  the  equidistant  points  provides  an  output  response  with 
more  or  less  variance  than  any  other  equidistant  point.  Although  rotatability  is  desirable,  it  also  is  not 
critical  to  the  success  of  the  empirical  modeling  effort  (Box  and  Draper,  1987:510). 

The  number  of  runs  required  by  an  experiment  depends  on  the  number  of  factors  involved  and 
the  design  implemented.  Using  as  few  runs  as  possible  is  desirable  because  it  saves  time  and  money. 
Ideally,  experimental  designs  would  be  orthogonal,  rotatable,  and  require  a  minimum  number  of  runs.  In 
practice,  resource  limitations,  operating  restrictions,  and  formal  policy  preclude  the  use  of  ideal 
experimental  designs.  In  spite  of  such  obstacles,  properly  designed  experiments  can  still  support 
empirical  model  building  and  exploitation. 

Types  of  Error 

Empirical  models  can  contain  two  types  of  errors:  random  and  bias  (Box  and  Draper,  1987:208). 
Random  error  results  when  it  is  possible  to  observe  different  output  responses  for  the  same  factor  level 
inputs;  that  is,  there  is  some  probabilistic  distribution  associated  with  the  output  response  for  a  set  of 
factor  level  inputs.  Random  error  results  from  some  uncontrollable  effect  on  the  process.  Bias  error  is  the 
difference  between  the  output  response  of  the  empirical  model  and  the  true  response  function  for  a  given 
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set  of  factor  level  inputs.  Bias  error  results  from  the  inability  of  the  empirical  model  to  fully  represent  the 
relationships  between  the  factors  and  output  response. 

Random  error  does  not  apply  to  RBL  because  its  underlying  solution  procedure,  METRIC, 
although  inherently  stochastic,  exhibits  “deterministic”  behavior  if  viewed  in  the  context  of  a  black  box. 

A  given  set  of  inputs  always  results  in  the  same  response,  regardless  of  the  number  of  runs  conducted. 

Thus,  the  error  present  in  an  empirical  model  of  RBL  is  completely  attributable  to  bias  — 
sometimes  referred  to  as  lack-of-fit.  With  no  random,  or  “pure”  error,  it  is  not  possible  to  measure  the 
quality  of  the  empirical  model  in  terms  of  statistical  significance.  The  adequacy  of  the  empirical  model 
must,  therefore,  be  measured  in  terms  associated  with  the  least-squares  fitting  procedure  such  as  R^, 
adjusted  R^,  and  mean  square  error. 

Canonical  Analysis 

If  the  resulting  empirical  model  is  a  second-order  polynomial,  the  description  of  the 
approximated  surface  can  be  made  more  straightforward  through  applying  a  canonical  analysis. 
“Canonical  analysis  is  a  method  of  rewriting  a  fitted  second  degree  equation  in  a  form  in  which  it  can  be 
more  readily  understood”  (Box  and  Draper,  1987:332).  Canonical  analysis  is  helpfully  applied  where 
factor  dependence  exists.  Factor  dependence  means  “the  response  function  for  one  factor  is  not 
independent  of  the  levels  of  the  other  factors”  (Box  and  Draper,  1987:329).  A  second  order  empirical 
model  —  before  canonical  analysis  --  exhibits  factor  dependence  by  the  presence  of  interaction  terms. 
Factor  dependencies  can  make  response  surface  description  difficult,  thus  it  is  by  elucidating  factor 
dependencies  that  canonical  analysis  aids  in  response  surface  description. 

If  the  empirical  model  is  more  readily  understood,  then  it  will  be  simpler  to  address  several 
possible  concerns.  Does  the  region  of  interest  also  happen  to  contain  an  extreme  point  (although 
searching  for  such  points  is  not  the  goal  of  this  research)?  If  an  extreme  point  exists,  is  it  a  maximum  or 
minimum?  Are  there  alternative  optima?  Is  it  possible  to  optimize  a  second  response  within  the  region  of 
interest?  Are  there  directions  of  insensitivity  with  respect  to  the  relationship  between  the  output  response 
and  factors?  Can  the  output  response  be  improved  by  simultaneous  changes  in  the  factor  levels?  Perhaps 


2-12 


a  basic  mechanism,  or  “natural  law,”  is  theorized  to  exist  between  two  or  more  factors.  Canonical 
analysis  could  increase  knowledge  about  the  existence  of  such  a  relationship  (Box  and  Draper, 
1987:329,330). 

There  are  two  canonical  forms:  A  and  B.  When  canonical  analysis  is  performed  in  practice,  the 
A  canonical  form  is  developed  first,  but  it  may  not  always  be  appropriate  to  develop  the  B  canonical  form. 
Briefly,  the  A  canonical  form  is  “achieved  by  a  rotation  of  axes  which  removes  all  cross-product  terms” 
(that  is,  interactions)  from  the  empirical  model  (Box  and  Draper,  1987:332).  The  A  canonical  form 
addresses  the  factor  dependence  discussed  previously. 

The  B  canonical  form  results  from  “a  change  of  origin  to  remove  first-order  terms...”  (Box  and 
Draper,  1987:332,333).  The  B  canonical  form  effectively  positions  the  empirical  model  over  the 
stationary  point  (i.e.  center  of  the  system  represented  as  an  extreme  point,  ridge,  or  saddle  point),  if  one 
exists.  The  B  canonical  form  is  appropriate  when  a  stationary  point  exists  and  is  close  (not  much  greater 
than  one  experimental-design  unit)  to  the  center  of  the  experimental  design  region  (Box  and  Draper, 
1987:339).  That  is,  the  movement  of  the  axes’  origin  should  not  go  beyond  the  experimental  design 
region. 

Summary 

This  chapter  discussed  the  process  under  study  -  RBL  -  and  the  elements  of  a  general 
methodology  -  RSM  -  for  developing  an  empirical  model  of  the  process.  RBL  considers  different  aspects 
of  the  multi-echelon  environment  as  it  takes  a  quantity  of  reparable  items,  one  type  at  a  time,  and  allocates 
them  between  the  bases  and  depot.  RBL  uses  marginal  analysis  to  determine  where  the  next  unit  should 
be  allocated.  Its  underlying  model,  METRIC,  was  discussed  via  its  purposes,  assumptions,  and  five-stage 
computational  process.  In  this  research,  only  the  first  four  stages  were  used  since  only  one  item  was 
analyzed. 

RSM  is  a  group  of  statistical  techniques  used  to  relate  various  process  inputs  to  a  process  output. 
The  output  may  have  an  analytical  or  theoretical  relationship  with  the  inputs.  The  true  relationship  is 
unknown  and,  therefore,  approximated  by  an  empirical  model.  The  empirical  model’s  response  surface 
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approximates  the  true  response  surface  within  a  limited  region.  RSM  uses  experimental  designs  to 
generate  the  data  required  to  develop  the  corresponding  empirical  models. 

Random  and  bias  errors  were  discussed,  but  random  error  was  of  no  concern  to  this  research 
because  of  the  “deterministic”  nature  of  the  METRIC  model.  Bias  error,  the  difference  between  the 
approximated  surface  and  the  true  surface,  was  a  concern  and  something  to  minimize.  Canonical  analysis 
facilitates  a  straightforward  interpretation  of  a  second-order  polynomial  model.  Through  axes  rotations 
and  translation,  canonical  analysis  can  highlight  factor  dependencies  and  the  principal  features  of  an 
empirical  model. 

Chapter  III  covers  the  details  of  applying  RSM  in  order  to  develop  an  empirical  model  for  the 
response  surface  of  RBL.  As  the  details  of  RSM  are  covered,  the  actual  analysis  of  RBL  takes  place.  The 
analysis  in  Chapter  III  should  be  taken  in  the  following  context:  this  paper  is  demonstrating  the  validity  of 
applying  RSM  to  RBL,  and  due  to  necessary  data  adjustments  mentioned  later,  any  conclusions  about 
correlations  between  the  output  and  factors  are  for  the  purpose  of  illustrating  interpretations  that  could  be 
made.  Chapter  IV  discusses  the  conclusions  drawn  from  the  analysis  in  Chapter  III  while  Chapter  V 
considers  studies  to  either  improve  or  extend  the  work  presented  in  this  paper. 


2-14 


III.  Analysis 


Introduction 

The  previous  chapter  stated  that  response  surface  methodology  (RSM)  could  develop  an 
empirical  model  of  the  readiness-based  leveling  (RBL)  technique.  The  goal  of  developing  an  empirical 
model  of  RBL  is  to  approximate  its  output,  or  response,  as  a  function  of  its  input  factors.  The  resulting 
empirical  model  can  then  be  analyzed  to  determine  the  significance  of  the  relationships  between  RBL’s 
output  response  and  input  variables,  or  factors.  This  study  required  a  reparable  item  to  provide  a  basis  for 
determining  factor  levels.  A  signal  processor,  National  Stock  Number  (NSN)  1270-01-396-6750WF,  was 
chosen  from  a  group  of  66  F-16  line-replaceable  units  (LRU)  because  of  its  high  activity  in  the  areas  of 
demand  rate,  worldwide  requirement,  and  expected  base  backorders  (EBO).  Appendix  A,  Table  A.  1,  lists 
the  values  for  these  criteria  for  all  of  the  66  LRUs  and  shows  the  ranking  procedure  through  which  the 
item  was  chosen  (Figure  A.  1).  The  data  sets  used  to  make  the  NSN  determination,  and  referenced  later  in 
this  chapter,  are  in  Appendix  B. 

Refer  to  the  List  of  Terms  for  a  definition  of  requirement.  The  requirements  data  set  in 
Appendix  B  includes  input  and  output  used  with  the  D041  Recoverable  Item  Requirements  System 
(henceforth  referred  to  as  D041).  D041  prioritizes  each  individual  item,  down  to  the  second  indenture 
level,  in  the  order  of  its  marginal  contribution  to  aircraft  availability.  Thus  D041  determines  the  optimal 
quantity  of  all  items  needed  to  maintain  the  desired  availability  of  a  particular  weapon  system.  The 
comprehensive  D041  solution  process  requires  a  great  deal  of  computing  resources. 

RBL  uses  the  D041  output  to  allocate  all  required  items  to  either  the  depot  or  one  of  the 
operational  bases.  Since  this  study  focused  on  only  one  type  of  item,  it  did  not  justify  the  computer 
resources  typically  consumed  by  D041.  Therefore,  a  method  was  needed  to  provide  a  substitute 
requirement  input  for  RBL.  Appendix  C  describes  the  method  used  in  this  study  to  directly  estimate  the 
requirement. 
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There  are  four  major  phases  to  this  analysis:  selecting  factors  and  factor  levels,  developing  and 
analyzing  a  first-order  empirical  model,  developing  and  analyzing  a  second-order  empirical  model  if 
necessary,  and  verifying  the  models. 

Factors  and  Factor  Levels 

Table  3.1  lists  the  seven  controllable  factors  used  in  this  research.  The  factors  are  controlled  in 
the  sense  that  their  values  were  purposely  chosen,  or  controlled,  in  an  experimental  setting.  Air  Force 
Materiel  Command  (AFMC)  Studies  and  Analysis  (SAO)  suggested  the  seven  factors  because  they  cover 
the  main  aspects  of  the  logistic  system  under  study,  and  they  are  presumed  to  be  influential  in  RBL’s 
computation  of  expected  base  backorders.  The  List  of  Terms  defines  each  factor.  Table  3.1  also  lists  the 
two  levels  at  which  the  factors  were  set.  Appendix  D  explains  why  the  levels  were  chosen.  The  levels 
shown  as  a  percentage  represent  a  percentage  decrease  (low  level)  or  increase  (high  level)  in  the  current 
level.  The  current  levels  were  collected  from  the  data  sets  described  in  Appendix  B.  The  term  “Vary”  in 
Table  3.1  refers  to  a  current  level  that  depends  on  the  base. 


Table  3.1  Controllable  Factors  and  Factor  Levels:  Model  Development 


FACTOR 

Low 

LEVEL 

Current 

High 

1.  Flying  hours/day  for  each  base  (FH) 

-20% 

vary 

+20% 

2.  Application  Percentage  (AP) 

-10% 

vaiy 

3.  Demand  rate  per  100  flying  hours  (DR) 

-30% 

1.0026 

+30% 

4.  Base  processing  Percentage  (BP) 

0.59 

0.77 

0.95 

5.  Base  repair  cycle  time  (BRCT) 

3 

5 

9 

6.  Order  and  ship  time  (OST) 

9 

9 

23 

7.  Depot  repair  cycle  time  (DRCT) 

20 

25 

30 

An  item’s  requirement  quantity  (REQ)  (as  calculated  in  Appendix  C)  was  also  considered  as  a  factor.  It  is 
not  shown  in  Table  3, 1  because  it  is  not  a  controllable  factor;  its  value  is  a  function  of  the  seven  factors 
listed  above  (see  Appendix  C). 
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First-Order  Strategy 

A  first-order  model  is  developed  through  a  first-order  strategy,  and  it  considers  only  the  main 
factors  as  possible  terms  in  the  model;  interaction  among  the  factors  and  quadratic  effects  are  not 
considered.  In  general,  the  choice  of  experimental  design  determines  the  types  of  models  that  may  be 
estimated. 

Every  design  has  associated  with  it  a  resolution,  which  is  usually  depicted  with  Roman  numerals; 
that  is,  IV  denotes  resolution  four.  “A  design  is  of  resolution  R  if  no  p-factor  effect  is  aliased  with  another 
effect  containing  less  than  R-p  factors”  (Montgomery,  1991:339).  For  example,  two-factor  effects  are  not 
aliased  with  one-factor  (or  main)  effects  in  a  resolution  IV  design,  but  they  are  aliased  with  two-factor  and 
higher  effects.  The  effects  of  “aliased”  factors  caimot  be  differentiated  from  one  another.  Designs  of 
resolution  III  or  IV  are  generally  used  to  develop  first-order  models,  whereas  second-order  models  require 
designs  of  at  least  resolution  V. 

In  this  research,  the  experimental  design  used  in  the  first-order  strategy  was  2’“^  vn-  This 
nomenclature  represents  64  runs  in  a  resolution  VII  design.  AFMC/SAO  suggested  three-factor 
interactions  were  the  highest  interactions  to  be  concerned  about.  With  a  resolution  VII  design,  three- 
factor  interactions  are  aliased  with  four-factor  and  higher  interactions.  Thus,  assuming  four-factor  and 
higher  interactions  are  not  significant,  there  is  no  confusion  attributable  to  aliasing  when  interpreting  the 
meaningfulness  of  three-factor  or  lower  interactions.  The  2^'^  vn  design  appears  with  the  RBL  outputs  in 
Appendix  E.  This  appendix  also  has  one  example  of  a  portion  of  a  design  point  (in  this  case  the  first 
design  point)  that  served  as  input  to  RBL. 

RBL’s  METRIC  model  is  stochastic  in  the  sense  that  there  is  a  probability  distribution  associated 
with  the  number  of  backorders  that  may  occur  at  any  point  in  time.  However,  this  stochastic  element  is 
represented  as  an  expected  value;  namely,  expected  base  backorders.  For  a  given  set  of  levels  for  each  of 
the  seven  controllable  factors,  calculations  in  Appendix  C  will  produce  a  requirement  quantity  that  serves 
as  input  to  METRIC  (along  with  five  of  the  controllable  factors).  METRIC  will  output  the  same  EBO 
value  for  a  given  set  of  inputs.  So,  although  it  incorporates  probability  distributions  for  backorders,  the 
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expected  backorder  quantity  will  not  vary  in  cases  of  repeated  inputs.  METRIC  is  not  a  deterministic 
model,  but  if  viewed  as  a  “black  box,”  it  may  seem  to  exhibit  such  a  characteristic.  For  this  reason,  the 
statistical  inferences  typically  associated  with  linear  regression  models  are  not  valid.  However,  there  are 
other  indicators  such  as  R^,  SSE,  and  MSE  that  indicate  the  quality  of  the  model  fit. 

Appendix  F  contains  information  about  the  experiments’  output  to  include  the  following: 
cumulative  distribution  functions  (CDF)  of  both  REQ  and  expected  back  orders  (EBO)  (Figures  F.  1,F.2), 
scatter  plots  of  each  factor  matched  against  EBO  (Figures  F.3  through  F.  10),  statistics  on  the  distributions 
of  REQ  and  EBO  (Table  F.  1),  statistics  on  each  of  the  controllable  factors  at  both  the  low  and  high  levels 
(Table  F.2),  and  a  correlation  matrix  (Table  F.3).  From  the  CDFs  and  skewness  statistic,  both  REQ  and 
EBO  show  evidence  of  skewness.  This  information  is  useful  later  for  considering  factor  transformations. 
In  the  scatter  plots  and  statistics  table,  the  levels  of  BP  and  DR  have  a  noticeable  effect  on  the  average 
EBO  value,  while  the  levels  of  BRCT  have  a  noticeable  effect  on  the  spread  of  EBO  values.  Noteworthy 
correlations  exist  between  REQ  and  EBO  and  BP  and  EBO.  Thus,  it  appears  supply  situations  calling  for 
a  low  item  requirement  are  correlated  with  low  EBO  values,  while  a  high  base  repair  percentage  is  also 
correlated  with  low  EBO  values.  DRCT  is  notable  for  its  lack  of  correlation  with  any  other  factor  or  the 
response. 

After  conducting  experiments  specified  in  the  design,  a  first-order  model  was  sought.  SAS 
software  was  used  for  a  portion  of  the  analysis  and  its  statistical  procedures  are  denoted  by  capital  letters. 
The  SAS  code  is  included  in  Appendix  G.  The  RSQUARE  procedure  used  the  R^  statistic  as  criteria 
while  seeking  the  best  one-factor  model,  the  best  two-factor  model,  and  so  on  until  all  eight  factors  were 
incorporated  into  a  model.  The  results  of  the  RSQUARE  procedure  are  included  in  Appendix  H.  The 
relationships  between  R^  and  MSE  and  the  number  of  factors  in  a  model  are  plotted  in  Figures  3.1  and 
3.2,  respectively.  Based  on  these  two  criteria,  Figures  3.1  and  3.2  indicate  models  with  three  to  five 
factors  as  being  roughly  equivalent. 

Recall  that  the  seven-factor  design  is  orthogonal.  The  zero  correlation  values  in  Table  F.3 
support  this  fact.  A  useful  result  of  the  orthogonal  design  is  the  coefficient  estimates  of  a  regression 
(incorporating  only  orthogonal  factors)  reflect  the  effect  each  factor  has  on  the  response.  Furthermore, 
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since  a  regression  coefiBcient  reflects  the  effect  on  the  response  of  a  one  unit  change  in  the  factor  (e.g. 
from  -1  to  0  or  0  to  1),  doubling  the  coefficient’s  value  reflects  the  effect  on  the  response  from  changing 
the  factor  level  between  -1  and  L  Figure  3.3  graphs  the  factor  effects  on  the  response.  It  is  the  difference 
between  the  average  EBO  value  when  a  factor  is  at  its  high  level  and  the  average  EBO  value  when  the 
factor  is  at  its  low  level.  The  first  three  factors  (BP,  DR,  FH)  have  the  greatest  impact  of  the  seven 
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Figure  3,1  vs.  Number  of  Model  Parameters 


Figure  3.2  MSE  vs.  Number  of  Model  Parameters 
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Figure  3.3  Effects  of  Controllable  Factors  on  EBO 


controllable  factors.  The  next  three  factors  (BRCT,  AP,  OST)  have  less  of  an  impact  and  are  similar  in 
magnitude.  REQ  is  not  included  in  this  graph  because  such  an  effect  cannot  be  determined;  REQ  is  not 
controllable  and  it  did  not  have  an  experimental  design  vector  of +1  and  -1.  Actually,  it  had  64  different 
“levels”  that  were  a  result  of  the  calculations  in  Appendix  C.  Thus,  it  is  not  possible  to  come  up  with  an 
effect  commensurate  to  those  in  Figure  3.3.  Furthermore,  when  REQ  is  added  to  a  regression  equation 
including  the  other  factors,  the  coefficients  and  intercept  term  change  due  to  the  correlation  exhibited  in 
Table  F.3  between  REQ  and  the  other  factors  and  the  EBO  response. 

Using  the  results  of  RSQUARE  as  a  starting  point,  five  first-order  models  were  developed.  The 
order  in  which  they  are  discussed  basically  follows  the  order  in  which  they  were  developed.  The  first 
model  developed  was: 

EBO  =  8. 114  + 0.7569  FH+  1.1509  DR  -  1.8519  BP  (3-1) 

With  an  R^  of  0.874,  it  incorporated  the  three  factors  in  Figure  3.3  with  the  largest  effects  on  EBO. 

Figure  3.4  shows  that  the  residuals  appear  to  increase  with  higher  predicted  values.  In  light  of  the  seven 
controllable  factors,  Equation  3-1  represents  the  most  parsimonious  model  that  was  developed.  At  the 
same  time,  its  factors  explain  the  influence  of  level  changes  on  EBO  well. 
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Figure  3.4  Residuals  vs.  Predicted  Values  for  Equation  3-1 


The  second  model  developed  was: 

EBO  -  8.114  +  0.7569  FH  +  0.3331  AP  +  1.1509  DR  -1.8519  BP  -  0.3482  BRCT  +  0.2965  OST  (3-2) 
With  an  R^  of  0.926,  it  is  the  opposite  extreme  on  the  “parsimony  scale,”  incorporating  six  factors.  Figure 
3.5  definitely  shows  a  pattern. 


Figure  3.5  Residuals  vs.  Predicted  Values  for  Equation  3-2 
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The  dispersion  of  the  residuals  from  zero  seems  to  be  related  to  the  magnitude  of  the  predicted  values. 
Such  patterns  usually  indicate  a  model  specification  error.  In  other  words,  there  is  a  structural  pattern  in 
the  EBO  response  that  is  not  modeled  by  the  terms  (factors  and  response)  of  Equation  3-2. 

The  graphical  and  correlation  relationships  demonstrated  in  Table  F.3  and  Figure  F.  10,  between 
the  REQ  factor  and  the  EBO  response,  strongly  suggest  including  REQ  in  an  empirical  model  of  EBO. 
The  RSQUARE  procedure  first  used  REQ  in  building  a  five-factor  model.  The  factor  was  excluded  from 
larger  models,  however,  until  it  reentered  in  the  eight-factor  model.  The  five-factor  model  including  REQ 
was: 

6.6010  +  0.4819  FH  +0.7384  DR  -  1.4758  BP  -  0.6922  BRCT  +  0.0099  REQ  (3-3) 
Its  R^  equaled  0.915  and  Figure  3.6  shows  a  less  severe  pattern  siitnlar  in  shape  to  Equation  3-2’s. 
However,  Figure  3.7  shows  somewhat  of  a  curving  pattern  between  the  residuals  and  REQ.  This  pattern 
may  call  for  a  transformation  on  the  REQ  values. 

Recall  the  small  presence  of  skewness  among  the  EBO  values  in  Figure  F.2.  Such  skewness  is  a 
cause  for  the  residual  patterns  present  in  Equations  3-2  and  3-3  (Montgomery,  1991:100).  Usually  a 
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Figure  3.6  Residuals  vs.  Predicted  Values  for  Equation  3-3 
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Figure  3.7  Residuals  vs.  REQ  for  Equation  3-3 


transformation  on  the  response  values  accommodates  the  skewness,  resulting  in  a  better-fitting  model.  A 
Box-Cox  transformation  analysis  transformed  the  EBO  response  iteratively,  seeking  a  transformed  EBO 
response  that  resulted  in  the  best  fit  for  Equation  3-3  (Montgomery,  1991:  105).  Figure  3.8  shows  a 
square-root  transformation  as  the  best  one  for  the  given  responses  and  model. 


Figure  3.8  Box-Cox  Transformation  Results 
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The  transformation  was  incorporated  into  the  fourth  model: 

=  2.6251  +  0.099  FH  +  0.1506  DR  -  0.2783  BP  -  0.092  BRCT  +  0.0013  REQ  (3-4) 
Its  R^  equaled  0.922  and  Figure  3.9  shows  the  transformation  has  eradicated  the  pattern.  However,  Figure 
3.10  still  shows  a  pattern  similar  to  the  one  for  Equation  3-3,  A  natural  logarithm  transformation  was 
used  to  compress  the  scale  of  the  REQ  values.  With  this  transformation  of  REQ,  the  Box-Cox 


Figure  3.9  Residuals  vs.  Predicted  Values  for  Equation  3-4 


Figure  3.10  Residuals  vs.  REQ  for  Equation  3-4 
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transformation  applied  in  Equation  3-4  is  no  longer  valid.  The  fifth  model  was  similar  to  Equation  3-3 
except  for  the  REQ  transformation: 

EBO  =  -7.214  + 0.1978 FH  + 0.2964 DR -0.9899 BP-  1.1777 BRCT  + 3.1341  InREQ  (3-5) 
Its  R^  equaled  0.956  and  Figure  3.11  shows  a  fair  plot  of  the  residuals.  Figure  3.12  shows  an  elimination 
of  the  pattern  that  was  present  in  Figures  3.7  and  3. 10. 


Figure  3.11  Residuals  vs.  Predicted  Values  for  Equation  3-5 


Figure  3.12  Residuals  vs.  In  REQ  for  Equation  3-5 
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Of  the  five  models  discussed  in  this  section,  Equations  3-1  and  3-5  are  noteworthy.  Between  the 
two.  Equation  3-1  is  more  parsimonious,  offers  a  straightforward  interpretation,  and  provides  good 
explanatory  capability.  It  shows  FH,  DR,  and  BP  to  be,  collectively,  the  most  influential  on  EBOs  of  the 
eight  factors  considered.  FH  and  DR  are  positively  correlated  with  EBO,  and  BP  is  negatively  correlated. 

Equation  3-5  offers  the  trade-off  between  a  slightly  more  cumbersome  model  and  a  higher  degree 
of  explanatory  capability.  FH,  DR,  and  BP  have  the  same  influences  on  EBO.  The  last  factor  in  Equation 
3-5,  REQ,  is  positively  correlated  with  EBO.  Interestingly,  BRCT  is  negatively  correlated  with  EBO. 
Intuitively,  it  may  not  make  sense  that  an  increase  in  the  amount  of  an  item’s  repair  time  is  correlated 
with  a  lower  EBO  value.  Soon  it  is  shown  the  interaction  effects  between  BRCT  and  the  other  factors 
cause  the  negative  sign.  Interaction  effects  can  be  investigated  with  the  first-order  design  output. 
Investigating  such  effects  can  reveal  relationships  among  factors  that  the  first-order  models  do  not 
consider. 

The  presence  of  interactions  means  the  response  surface  has  curvature.  Twenty-one  two-way 
interactions  were  possible  between  the  seven  controllable  factors.  The  ensuing  analysis  was  not  applicable 
to  REQ  for  the  same  reasons  its  effect  on  EBO  could  not  be  derived  like  it  was  for  the  other  factors. 
Appendix  I  presents  the  interactions.  The  MSE  of  the  plotted  average  effects  and  the  factors’  scales  may 
influence  the  degree  of  interaction  present.  An  interaction  effect  exists  if  a  factor’s  effect  on  the  EBO 
depends  on  the  level  of  another  factor.  For  example,  see  Figure  3. 12.  Initially  consider  BRCT  at  the  high 
level  (+1).  FH’s  effect  on  EBO  as  FH’s  level  changes  from  -I  to  +I  is  approximately  one.  Now  consider 
BRCT  at  the  low  level  (-1).  FH’s  effect  on  EBO  as  FH’s  level  changes  from  -1  to  +1  is  approximately 
two.  Therefore,  the  effect  FH  has  on  EBO  depends  on  the  level  of  BRCT;  one  if  BRCT  is  high,  two  if 
BRCT  is  low.  Other  noteworthy  interactions  are  depicted  between  BRCT  and  DR,  BP,  and  OST  in 
Figures  1. 12, 1. 15,  and  1. 18,  respectively. 

The  interactions  between  BRCT  and  the  other  factors  in  the  model  (FH,  DR,  BP)  provides  an 
explanation  for  the  counterintuitive  negative  correlation  between  EBO  and  BRCT  in  Equation  3-5.  In 
Equation  3-5,  a  low  level  of  FH  is  correlated  with  a  lower  quantity  of  EBO.  Reconsider  Figure  3,12  and 
FH  at  its  low  level.  With  FH  at  its  low  level,  the  high  level  of  BRCT  corresponds  to  the  lower  EBO  value. 
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Equation  3-5,  a  low  level  of  FH  is  correlated  with  a  lower  quantity  of  EBO.  Reconsider  Figure  3.12  and 
FH  at  its  low  level.  With  FH  at  its  low  level,  the  high  level  of  BRCT  corresponds  to  the  lower  EBO  value. 


FH  level 

Figure  3.13  Interaction  Between  FH  and  BRCT 

Since  BRCT  is  negatively  correlated  with  EBO,  a  high  level  of  BRCT  would  certainly  contribute  to  lower 
EBO  as  far  as  the  regression  is  concerned.  The  same  reasoning  holds  true  between  DR  and  BRCT  in 
Figure  1. 12  in  Appendix  I.  The  reasoning  does  not  hold  up  between  BP  and  BRCT  in  Figure  1. 15,  but  the 
EBO  values  are  so  close  anyway.  Although  the  interactions  illustrate  the  role  of  BRCT’ s  negative  sign,  it 
does  not  resolve  the  counter-intuitiveness. 

The  presence  of  interactions  suggest  that  the  response  surface  of  EBO  exhibits  a  degree  of 
curvature.  Although  the  fitted  models  might  be  suitable  empirical  models  of  RBL,  the  addition  of 
quadratic  terms  offers  the  possibility  of  improving  their  explanatory  capability. 

Second-Order  Strategy 

The  original  factorial  design  points  were  augmented  in  order  to  estimate  quadratic  terms.  The 
result  was  a  face-centered  central  composite  design  (FCCCD)  (Montgomery,  1991:546).  In  addition  to 
the  original  two-level  factorial  design,  the  FCCCD  included  a  center  point  and  two  axial  points  for  each 
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factor.  The  three  levels  for  each  of  the  factors  in  the  FCCCD  were  necessary  for  developing  a  quadratic 
model.  One  advantage  of  a  FCCCD  is  that  it  “fits”  within  the  design  space  of  the  original  two-level 
design  where  the  design  settings  are  already  at  their  extremes. 

The  FCCCD  design  is  not  rotatable  because  the  axial  points  stay  within  the  two-level  design 
space.  The  number  of  center  points  determine  the  orthogonality  of  the  augmented  design.  For  a  2^’^  vn 
design,  22  center  points  are  required  for  orthogonality  (Montgomery,  1991:546).  With  the  METRIC 
model,  multiple  center  points  do  not  provide  any  new  information:  for  a  unique  set  of  inputs,  the  same 
output  occurs.  Just  one  center  point  provides  all  the  information.  Table  3.2  summarizes  the  difference 
between  a  design  with  22  center  points  and  a  design  with  one  center  point;  all  14  axial  points  are 
included. 


Table  3.2:  Impact  of  Center  Points  on  FCCCD 


Design 

Sqrt  MSE 

Response 

Mean 

Coefficient 

of 

Variation 

(%) 

R^for 

Linear 

Terms 

R^for 

Quadratic 

Terms 

R^for 

Interaction 

Terms 

0.9988 

0.0955 

8.4061 

1.14 

0.8981 

0.0603 

0.0404 

0.9988 

0.1191 

8.2503 

1.44 

0.9179 

0.0396 

0.0413 

Table  3.2  shows  an  increase  in  MSE  and  variation  as  the  number  of  center  points  decreased  from  22  to 
one,  but  the  amounts  are  minimal.  The  second-order  strategy  used  a  design  with  one  center  point  because 
it  made  sense  as  far  as  the  information  it  has  to  contribute,  and  the  impact  of  the  missing  21  center  points 
is  minimal. 

Typically,  center  points  are  generated  before  axial  points  in  order  to  determine  whether  or  not 
there  is  curvature  in  the  response  surface  (that  is,  to  test  for  a  quadratic  effect)  (Schmidt  and  Launsby, 
1989:322),  However,  investigation  of  the  interaction  effects  with  the  first-order  design  has  already  shown 
the  presence  of  curvature.  Therefore,  the  center  and  axial  points  were  incorporated  into  the  FCCCD 
simultaneously  in  order  to  estimate  all  second-order  terms.  The  augmented  design  points  and  their 
corresponding  RBL  responses  are  shown  in  Appendix  J. 
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Appendix  K  contains  information  for  the  79  experiment’s  outputs  similar  to  that  shown  in 
Appendix  F.  The  mean  and  median  values  of  REQ  and  EBO  increased  slightly,  while  the  skewness 
increased  for  REQ  and  decreased  for  EBO  (Table  K.  1).  Table  K.2  contains  average  response  values  for 
the  center  points.  The  statistics  on  each  of  the  controllable  factors  at  both  the  low  and  high  values  were 
veiy  similar  to  those  for  the  factorial  data  set.  The  correlation  matrix  in  Table  K.3  shows  two  notable 
relationships  that  were  present  in  Table  F.3  as  well:  BP  and  EBO  (-0.74),  REQ  and  EBO  (0.70).  The 
scatter  plots  in  Figures  K.3  through  K.IO  do  not  show  any  evidence  of  quadratic  effect  except  for  REQ  in 
Figure  K.  10. 

After  conducting  experiments  specified  in  the  design,  a  second-order  model  was  sought.  The 
SAS  code  is  included  in  Appendix  L.  As  was  done  for  the  first-order  strategy,  procedure  RSQUARE  was 
run  on  the  44  terms  and  its  results  are  in  Appendix  M.  Figures  3.14  and  3.15  plot  the  relationships 
between  R^  and  MSE  against  the  number  of  factors  in  a  model.  A  three  or  four-factor  model  (certainly  no 
more  than  five)  appeared  to  be  adequate  in  terms  of  explanatory  capability.  A  plot  of  factor  effects  similar 
to  Figine  3.3  cannot  be  shown  because  of  the  presence  of  REQ  in  all  of  the  models  except  the  one-factor 
model.  Recall  REQ  is  not  orthogonal  to  the  controllable  factors,  therefore  parameter  estimates  may 
change  as  the  models  incorporating  REQ  change. 


Figure  3.14  R-square  vs.  Number  of  Model  Parameters 
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Figure  3.15  MSE  vs.  Number  of  Model  Parameters 


SAS  procedure  RSREG  (response  surface  regression)  generated  a  model  with  all  linear, 
interaction,  and  quadratic  terms:  44  terms  total,  not  including  the  intercept  (see  Appendix  N).  The 
equaled  0.999.  The  possibility  of  meaningful  three-way  and  higher  interactions  is  practically  nonexistent 
because  the  error  sum  of  squares  is  almost  zero.  However,  a  44-term  model  is  not  parsimonious. 

Two  second-order  models  were  analyzed  with  three  and  four  factors.  They  were  generated  by  the 
RSQUARE  procedure;  these  two  models  are  the  best  three-factor  and  four-factor  models  in  terms  of  R^. 
The  three-factor  model  was: 

EBO  =  4.8183  -  0.8805  BP  +  0.0239  REQ  -  0.008  BRCT*REQ  (3-6) 

Its  R^  equaled  0.925  and  Figure  3.16  shows  no  strong  pattern.  Figure  3. 17  shows  a  curving  pattern 
similar  to  the  one  experienced  in  the  first-order  models,  suggesting  a  needed  transformation. 

The  four-factor  model  was: 

EBO  =  2.535  +  0.0522  REQ  -  0.0053  BP*REQ  -  0.0077  BRCT*REQ  -  0.00008  REQ^  (3-7) 
Its  R^  equaled  0.961  and  Figures  3.18  and  3. 19  show  good  scatter  plots  with  an  eradication  of  the  curving 
pattern  in  the  REQ  plot. 
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Figure  3.16  Residuals  vs.  Predicted  Values  for  Equation  3-6 


Figure  3.17  Residuals  vs.  REQ  for  Model  3-6 


In  comparing  all  seven  models,  a  couple  stand  out.  Equation  3-6  is  more  parsimonious  than  any 
other  model  except  for  Equation  3-1  however,  for  the  same  levels  of  parsimony.  Equation  3-6  has  a  larger 
than  Equation  3-1.  Equation  3-7  has  the  highest  plus  it  has  one  less  factor  than  the  model  with  the 
next  highest  R^,  Equation  3-5.  The  transformations  in  Equations  3-4  and  3-5  do  not  aJEFord  factor  effect 
interpretations  as  straightforward  as  Equation  3-6,  which  has  no  transformations.  Furthermore,  models 
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Figure  3.18  Residuals  vs.  Predicted  Values  for  Equation  3-7 


Figure  3.19  Residuals  vs.  REQ  for  Equation  3-7 


without  transformations  are  simpler  to  use  because  they  do  not  require  additional  computation  time  to 
perform  transformations.  Although  Equation  3-7  has  a  transformation,  it  can  be  easily  interpreted  as  the 
main  source  of  curvature  in  the  RBL  response  surface.  In  light  of  this  discussion  on  parsimony, 
explanatory  capability,  and  interpretability.  Equation  3-6  appears  to  have  the  best  combination  of  these 
three  model  attributes. 
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A  canonical  analysis  was  performed  in  an  effort  to  provide  additional  information  on  the  surface 
curvature  within  the  design  region.  Appendix  O  explains  the  development  of  the  A  canonical  form  shown 
here: 

EBO  =  9.8356  +  0.5046  FH  -  0.3908  AP  +  0.7903  DR  -  0.9595  BP  +  0.5559  BRCT  + 

2.0464  OST  +  0.6450  DRCT  -  0.01 17  REQ  +  0.2839  FH^  +  0.0473  AP^  -  0.041 1  DR^  -  (3.8) 

0.0592  BP^  -  0. 1276  BRCT^  -  0.2647  OST^  -  0.4368  DRCT^  -  0.9197  REQ^ 

Two  points  can  be  made  from  Equation  3-8.  First,  by  virtue  of  its  coefficient’s  magnitude,  REQ  has  a 
large  linear  correlation  with  EBO.  With  an  approximate  mean  of  153  in  the  second-order  design,  on 
average  REQ  affects  EBO  by  approximately  -1.7  backorders.  Second  and  similarly,  since  REQ^  ‘s 
coefficient  is  the  largest  of  the  quadratic  terms,  requirement  plays  the  largest  role  in  shaping  the  response 
smface. 

Interaction  and  quadratic  effects  have  shown  to  be  useful  in  developing  models  that  are  more 
explanatory  than  the  first-order  models  while  at  the  same  time  more  parsimonious.  However,  the 
explanatory  capability  of  the  first  and  second-order  models,  as  given  in  the  R^  statistic,  only  applies  to  the 
experimental  data  set.  Using  a  verification  data  set  provides  a  means  for  determining  the  flexibility  of  the 
model  to  explain  behavior  when  new  design  points  are  considered. 

Model  Verification 

The  verification  process  proceeded  in  three  steps: 

1.  Different  factor  levels  were  chosen  and  coded. 

2.  The  coded  levels  were  ran  in  RBL  according  to  the  second-order  design. 

3.  A  comparison  was  made  between  RBL’s  observed  EBO  responses  and  the  empirical  models’  predicted 
responses. 

The  new  factor  levels  were  within  the  design  space  but  not  at  levels  used  in  the  development  of  the  first 
and  second-order  models.  These  design  points  tested  a  model’s  accuracy  in  areas  of  the  response  surface 
where  no  previous  information  (via  RBL  runs)  existed.  Table  3.3  shows  the  new  factor  levels  for  the 
controllable  factors. 
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Table  3.3  Controllable  Factors  and  Factor  Levels:  Model  Verification 


FACTOR 

Low 

LEVEL 

Center 

High 

1 .  Flying  hours/day  for  each  base  (FH) 

-18% 

-5% 

+10% 

2.  Application  Percentage  (AP) 

-5% 

current 

+5% 

3.  Demand  rate  per  100  flying  hours  (DR) 

-15% 

+5% 

+15% 

4.  Base  processing  Percentage  (BP) 

0.67 

0.75 

0.85 

5.  Base  repair  cycle  time  (BRCT) 

5 

6 

8 

6.  Order  and  ship  time  (OST) 

10 

13 

19 

7.  Depot  repair  cycle  time  (DRCT) 

23 

27 

29 

Three  measures  of  empirical  model  performance  were  used.  First,  for  each  design  point,  the 
difference  between  the  observed  RBL  output  and  an  empirical  modeFs  predicted  value  was  calculated, 
resulting  in  a  prediction  error.  The  ratio  of  this  difference  and  the  observed  value  provides  a  measure  of 
error  as  a  percentage  of  the  observed  value.  Second,  the  predicted  EBO  values  were  plotted  against  the 
observed  EBO  values.  A  high  correlation  pattern  indicated  good  model  performance.  Third,  the  stability 
of  the  empirical  models’  parameters  were  investigated.  That  is,  for  an  existing  model’s  particular  set  of 
factors,  the  model  was  refit  to  the  verification  data  set.  If  the  new  model’s  parameter  estimates  were  very 
similar  to  the  original  model,  then  confidence  in  the  original  model’s  explanatory  capability  is  justified 
since  basically  the  same  model  explained  the  response  surface  in  different  regions.  These  three  measures 
were  applied  to  all  seven  models  previously  developed. 

The  uncoded  design  points  and  RBL  output  are  shown  in  Appendix  P,  Table  P.  1  for  the  factorial 
points,  and  Table  P.2  for  the  center  and  axial  points.  Table  3.4  contains  information  for  the  verification 
runs’  outputs  similar  to  that  shown  in  Appendices  F  and  K.  Compared  to  the  second-order  design,  the 
new  verification  design  points  resulted  in  higher  EBO  and  REQ  values  with  almost  half  the  standard 
deviations.  REQ  and  EBO  have  less  skew  which  is  evident  not  only  in  their  skew  statistics  but  also  in  the 
close  mean  and  median  values.  The  correlation  matrix  in  Table  P.3  shows  two  notable  relationships  that 
were  present  in  Tables  F.3  and  K.3  as  well:  BP  and  EBO  (-0.657),  REQ  and  EBO  (0.736). 
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Table  3.4  REQ  and  EBO  Distribution  Statistics 


REQ 

EBO 

Mean 

152.835 

8.573 

Median 

152.000 

8.618 

Std.  Dev. 

37.313 

1.174 

Skewness 

0.650 

0.160 

Kurtosis 

0.608 

-0.198 

Once  the  verification  runs  were  complete,  the  first  of  the  three  measures  of  empirical  model 
performance,  percentage  of  error,  was  calculated.  Table  3.5  shows  the  average  error  percentage  along 
with  other  performance  statistics  for  all  of  the  models.  Model  3-7  sustained  the  least  amount  of  error 
against  the  verification  data  set.  The  second  measure  of  empirical  model  performance,  correlation  of  EBO 
values,  is  also  shown  Table  3.5,  and  Model  3-7  had  the  highest  value  for  this  measure  as  well.  Figure 
3.20  shows  the  high  degree  of  correlation  of  Model  3-7  graphically. 
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Figure  3.20  Predicted  Values  vs.  Observed  Values  for  Model  3-7 
The  third  measure  of  empirical  model  performance  was  the  stability  of  the  models’  parameters.  Table  3.6 
shows  the  two  regression  equations  for  each  model;  one  for  the  original  data  set  and  one  after  refitting  the 
model  to  the  verification  data  set.  None  of  the  models’  parameter  signs  changed  between  regressions  on 
both  data  sets.  The  amount  of  change  in  the  parameters’  magnitudes  was  not  large;  Model  3-7’s 
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parameters  are  the  most  stable.  Since  the  parameters’  values  did  not  change  by  a  large  amount, 
confidence  can  be  placed  in  Model  3-7’s  original  parameter  estimates.  Model  3-7’s  performance  should 
not  unduly  overshadow  the  other  models’  performances;  all  models  performed  well. 

Summary 

This  chapter  has  shown  how  response  surface  methodology  can  be  applied  to  characterize  a 
response  function  for  the  RBL  technique.  Several  empirical  representations  of  the  response  surface  were 
developed.  The  confidence  in  these  representations  ranged  from  fairly  high  to  very  high.  By  exploring 
several  models  with  different  levels  of  parsimony,  the  influences  of  different  factors  on  EBO  could  be 
investigated.  The  high  explanatory  capability  (R\  which  all  models  possessed,  substantiated  the 
influences  of  factors  within  that  model. 

The  method  of  applying  RSM  is  emphasized  since  the  scope  of  this  research  precluded  the  use  of 
the  computationally  intensive  D041  model.  Therefore,  any  statements  made  about  factor  influences  must 
be  taken  in  the  context  of  first,  a  notional  D041  requirement  input  and,  second,  the  design  space  defined 
by  the  extreme  factor  levels  of  the  first-order  design.  Otherwise,  the  steps  taken,  analyses  conducted,  and 
interpretations  made  in  this  study  are  typical  of  a  classic  RSM  study  of  RBL. 
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IV.  Conclusions 


The  purpose  of  this  research  was  to  demonstrate  that  response  surface  methodology  (RSM)  could 
be  used  to  explore  possible  relationships  between  some  of  the  factors  affecting  readiness-based  leveling 
(RBL)  and  its  output  response,  expected  backorders  (EBO).  Empirical  models  were  developed  based  on 
statistics  of  goodness  of  fit,  such  as  and  MSE,  and  an  analysis  of  the  residuals.  By  virtue  of  the  models 
developed  and  their  level  of  explanatory  capability,  the  demonstration  is  complete.  Throughout  this 
research,  two  issues  related  to  developing  and  interpreting  the  response  surface  models  consistently 
emerged. 

First,  the  use  of  the  Recoverable  Item  Requirements  System  (D041)  model  to  generate  the  spares 
requirement  was  unwarranted  for  the  scope  and  purpose  of  this  research.  Whereas  D041  prioritizes  each 
individual  item,  down  to  the  second-indenture  level,  in  the  order  of  its  marginal  contribution  to  aircraft 
availability,  this  research  focused  on  one  first-indentured  item.  Unfortunately,  in  practice  D04rs  output 
is  the  RBL  input.  Since  a  substitute  D041  input  was  used,  any  specific  conclusions  concerning  the  effect 
of  the  factors  investigated  on  EBO  are  somewhat  suspect.  Not  until  the  methodology  demonstrated  in  this 
paper  is  applied  with  data  used  in  practice  can  valid  conclusions  regarding  those  relationships  be  made. 

Second,  the  model  at  the  heart  of  RBL,  Multi-Echelon  Technique  for  Recoverable  Item  Control 
(METRIC),  exhibits  a  ‘‘deterministic”  behavior  if  it  is  viewed  as  a  “black  box;”  a  given  set  of  inputs 
always  yields  the  same  output.  Because  of  this  behavior,  replications  of  any  designs  points  served  no 
purpose.  Usually,  replications  are  performed  in  order  to  estimate  pure  error  in  a  process,  but  METRIC 
does  not  have  pure  error.  Hence,  all  of  the  models’  prediction  error  is  due  to  lack  of  fit. 

The  behavior  of  all  seven  models  in  this  study  was  such  that  no  new  interpretations  of  factor 
influences  had  to  be  made  between  one  model  and  another.  Provided  the  models  develop  in  a  similar 
manner  using  the  D041  input,  the  insight  gained  will  have  a  degree  of  commonality  among  the  models.  If 
the  models  do  not  develop  in  such  a  manner,  determining  the  “best”  model  to  represent  the  process  in 
question  will  require  more  thought.  It  will  be  this  model  which  supports  conclusions  about  factor 
influences  in  RBL. 
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V.  Considerations  for  Further  Study 


As  response  surface  methodology  (RSM)  is  by  nature  an  iterative  process,  the  line  of  research 
begun  in  this  paper  is  by  no  means  complete.  This  chapter  will  highlight  new  perspectives  for  further 
RSM  analysis  on  Readiness-Based  Leveling  (RBL). 

The  first  new  perspective  would  address  the  fact  that  a  substitute  D041  requirement  was  used  as 
an  RBL  input.  Because  the  actual  D041  requirement  values  were  not  incorporated  in  this  study,  definitive 
conclusions  concerning  the  effect  of  the  factors  investigated  on  expected  backorders  (EBO)  were  not 
made.  Certain  factors  were  highlighted  as  having  large  effects  on  RBL’s  output,  but  only  for  the  sake  of 
demonstrating  the  kinds  of  observations  that  could  be  made  in  an  RSM  study.  Thus,  RSM  should  be 
applied  to  the  RBL  technique  with  the  D041  requirement  values  incorporated.  If  the  factor  effects  happen 
to  be  similar  to  those  shown  in  this  study,  then  the  models  developed  in  this  research  gain  more 
credibility.  Furthermore,  a  comparison  could  be  made  between  the  requirements  generated  in  this  study 
and  the  requirements  generated  by  the  D041  model;  high  correlation  would  tend  to  validate  the 
requirement  calculations  performed  in  this  study. 

The  second  new  perspective  would  consider  applying  RSM  to  other  resource-leveling  techniques. 
There  are  other  ways  to  allocate  reparable  items  between  the  bases  and  the  depot;  for  instance,  one 
thought  is  to  keep  a  certain  or  minimal  amount  of  stock  at  the  depot  with  the  remainder  allocated  between 
the  bases.  If  this  leveling  technique  was  analyzed,  then  the  resulting  factor  influences  could  be  compared 
to  the  analysis  described  in  the  preceding  paragraph. 

The  third  new  perspective  would  attempt  to  modify  the  design  region  under  study  by  extending 
the  extreme  factor  level  values.  This  modification  should  not  be  done  without  sufficient  rationale.  For 
instance,  the  item  used  in  this  study  had  a  specified  demand  rate,  and  a  degree  of  change  below  and  above 
this  demand  rate  denoted  the  extreme  factor  levels.  To  widen  the  difference  between  the  extreme  values 
for  this  item,  just  to  modify  the  design  space,  should  not  be  done  unless  considered  realistic.  There  are 
other  ways  to  modify  the  design  region.  For  example,  all  data  in  this  study  applied  to  bases  and  units 
flying  the  F- 16  aircraft.  If  other  weapon  systems  were  considered,  each  might  have  a  unique  response 
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surface.  Other  examples  include  the  consideration  of  second-indentured  items  or  items  that  belong  to 
different  weapon  subsystems. 

The  fourth  new  perspective  would  consider  base  and  depot  safely  levels  as  controllable  factors. 
These  two  factors  would  be  in  addition  to  the  seven  controllable  factors  and  one  uncontrollable  factor 
included  in  this  study.  Thus,  the  two  major  components  of  the  requirement  calculation,  pipelines  and 
safety  levels,  are  directly  controllable  within  the  experimental  environment.  Incorporating  safety  level 
factors  would  allow  an  analysis  of  their  influence  on  expected  backorders.  For  example,  if  it  is  believed 
that  the  depot  repair  pipeline  could  be  controlled  in  the  field  and  decreased,  then  less  items  would  be 
required  in  that  pipeline.  A  certain  number  of  items  would  then  become  available  as  safety  stock. 
Theoretically,  increasing  the  safety  stock  should  result  in  better  aircraft  availability.  In  the  context  of  an 
empirical  model  incorporating  the  pipelines  and  safety  levels  of  interest,  the  changes  in  pipelines  and 
safety  levels  could  be  imputed  into  the  model  and  a  predicted  expected  backorder  quantity  computed. 

Whichever  new  perspective(s)  may  be  taken,  the  validity  of  applying  RSM  to  RBL  remains  and 
the  result  is  a  response  surface  reflecting  how  inputs  are  affecting  an  output.  The  ‘"how”  is  manifested  in 
the  resulting  empirical  model.  The  “why”  behind  relationships  between  inputs  and  output  cannot  be 
garnered  from  an  empirical  model.  Instead,  such  a  question  is  answered  through  a  mechanistic  model: 
one  that  reflects  the  true  relationships  and  is  incapable  of  error.  Natural  laws  are  such  models.  Even 
though  an  empirical  model  is  not  the  mechanistic  model,  such  modeling  presents  a  structured,  systematic 
approach  to  learning  more  about  a  process;  in  this  case,  readiness-based  leveling  of  reparable  items. 
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Appendix  A 

National  Stock  Number  Selection 


Table  A.  1  shows  the  ranking  by  national  stock  number  (NSN)  of  66  F-16  line-replaceable  units  against 
three  criteria:  expected  base  backorders  (EBO),  requirement  (REQ),  demand  rate  (DR).  The  table 
continues  on  the  next  page.  On  the  third  page.  Figure  A.  1  explains  the  method  for  choosing  an  NSN. 


Table  A.1  NSN  Ranking  on  Three  Criteria 


Rank 

NSN 

EBO 

NSN 

REQ 

NSN 

DR 

1 

598501 21 22950WF 

13.3252 

598501 21 22950WF 

168 

1280012804855WF 

1.2464 

2 

127001 3590877WF 

10.2277 

1270012383662WF 

147 

127001 1022965WF 

1.0536 

3 

589501 1420803WF 

9.5443 

127001 3966750WFI 

135 

1  127001 3966750WFI 

1.0026 

4 

127001 3851 879WF 

9.481 

127001233001 1WF 

115 

127001 3590877WF 

0.9111 

5 

6625011 938861 WF 

8.8804 

594501 1709363WF 

115 

127001 1022966WF 

0.8544 

6 

1270992512706WF 

8.5395 

127099251 2706WF 

95 

5895012489012WF 

0.8 

7 

1 27001 3333608WF 

8.3767 

6615013619746WF 

84 

127001 1022962WF 

0.7663 

8 

1 27001 2383662WF 

8.0889 

5895012489012WF 

83 

660501 0876645WF 

0.7405 

9 

1 1 27001 3966750WFI 

7.6478 

126001 251 11 50WF 

76 

127001 1022963WF 

0.7375 

10 

1 27001 1022965WF 

7.5499 

1290012279260WF 

67 

589501 1420803WF 

0.6342 

11 

6605993708249WF 

6.6578 

1 27001 2352370WF 

66 

589501 355841 4WF 

0.6255 

12 

127001 1022966WF 

6.5061 

660501 0876645WF 

61 

1270012223829WF 

0.5596 

13 

1 27001 0932256WF 

6.482 

6625011 938861  WF 

58 

127099251 2706WF 

0.5528 

14 

126001 351 0592WF 

6.2798 

589501 1420803WF 

51 

127001 3093077WF 

0.431 

15 

127001 1022963WF 

5.823 

1270992255327WF 

47 

1270012122990WF 

0.4205 

16 

1 27001 3093077WF 

5.6941 

127001 3093077WF 

43 

1 29001 2279260WF 

0.4191 

17 

1 29001 2279260WF 

4.9667 

599901 0803978WF 

43 

1270012352370WF 

0.4186 

18 

126001251 1150WF 

4.5987 

661 001 3081 859WF 

43 

1270992255327WF 

0.4172 

19 

598501 3083647WF 

4.4767 

599801 3227746WF 

40 

598501 2935451  WF 

0.4019 

20 

599801 3309073WF 

4.2984 

126001 351 0592WF 

39 

127001 0932256WF 

0.388 

21 

129001 340631 7WF 

4.0771 

129001 3765449 WF 

39 

129001 340631 7WF 

0.3832 

22 

661 001 3081 859WF 

3.8963 

6615011273160WF 

38 

6605993708249WF 

0.3733 

23 

1 29001 3765449WF 

3.8878 

127001 3333608WF 

37 

598501 3083647WF 

0.3348 

24 

589501 355841 4WF 

3.5301 

589501 1435443WF 

37 

127001 3851 879WF 

0.3335 

25 

1270992255327WF 

3.4794 

6605993708249WF 

37 

6615013619746WF 

0.3311 

26 

1270012122990WF 

3.142 

6615013517337WF 

36 

660501 0463533WF 

0.3292 

27 

127001 282791 4WF 

3.0281 

127001 3590877WF 

35 

1 27001 2383662WF 

0.3178 

28 

5985012935451WF 

3.0071 

129001 322371 1WF 

34 

661 501 3566851  WF 

0.3084 

29 

127001 1022962WF 

2.9835 

660501 0463533WF 

34 

129001322371 1WF 

0.3078 

30 

1270012352370WF 

2.8766 

589501 2301 075WF 

33 

129001 3765449WF 

0.3078 

31 

1270012223829WF 

2.8253 

127001 3851 879WF 

31 

126001 351 0592WF 

0.2957 

32 

5841010963945WF 

2.8195 

589501 355841 4WF 

31 

127001 3333608WF 

0.2875 

33 

129001322371 1WF 

2.8177 

127001 0932256 WF 

30 

1270011 336494 WF 

0.2873 

34 

599901 0803978WF 

2.7098 

6615013246374WF 

30 

126001251 1150WF 

0.2635 

35 

660501 0463533WF 

2.467 

5841010963945WF 

29 

599801 1230046WF 

0.2627 

36 

5841010964833WF 

2.3508 

6615010427834WF 

29 

1270012827914WF 

0.2512 

37 

6615013246374WF 

2.3282 

5841010964833WF 

28 

127001 0453976WF 

0.2426 
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Table  A.1  (continued) 


Rank 

NSN 

EBO 

NSN 

REQ 

NSN 

DR 

38 

589501 2301 075WF 

2.1919 

599801 3309073WF 

28 

6615013528570WF 

0.2375 

39 

599801 1230046WF 

2.1604 

1270012122990WF 

26 

598501 21 22950WF 

0.2305 

40 

1 28001 2804855WF 

2.0456 

598501 31 64588WF 

26 

5841010964833WF 

0.2167 

41 

6615011 2731 60WF 

2.0357 

6615011496398WF 

26 

599801 3309073WF 

0.2146 

42 

599801 3227746WF 

2.0147 

661501 1297445WF 

24 

5841010963945WF 

0.2127 

43 

589501 248901 2WF 

1.9696 

127001 1022962WF 

22 

127001233001 1WF 

0.1721 

44 

589501 1435443WF 

1.9245 

129001 340631 7WF 

21 

6615013246374WF 

0.1706 

45 

6615010427834WF 

1.8722 

1 27001 2223829WF 

19 

589501 2301 075WF 

0.1511 

46 

660501 0876645WF 

1.7211 

127001 1022966WF 

17 

598501 31 64588WF 

0.135 

47 

1 27001 1336494WF 

1.6615 

599801 1230046WF 

17 

661501 1273160WF 

0.1196 

48 

1 27001 0946872WF 

1.5834 

127001 1022965WF 

16 

661 001 3081 859WF 

0.1175 

49 

661 501 3566851 WF 

1.4984 

127001 282791 4WF 

16 

661 001 37281 70WF 

0.1175 

50 

1280011 21 6879WF 

1.4733 

661 001 039781 7WF 

16 

129001 0800203WF 

0.0877 

51 

1270012330011WF 

1.3265 

598501 2935451  WF 

14 

6625011 938861  WF 

0.0816 

52 

6615013619746WF 

1.2779 

129001 0800203WF 

13 

589501 1435443WF 

0.0727 

53 

1 29001 0800203WF 

1.264 

127001 1022963WF 

12 

6615013517337WF 

0.0723 

54 

6615011297445WF 

1.053 

5821 01 0771 31 3WF 

12 

128001 121 6879WF 

0.0721 

55 

5821 01 0771 31 3WF 

0.9839 

661 501 3566851  WF 

12 

598501 31 64589WF 

0.054 

56 

661 001 039781 7WF 

0.9065 

1280011 21 6879WF 

11 

127001 0946872WF 

0.0522 

57 

598501 31 64588WF 

0.8477 

598501 3083647WF 

11 

599901 0803978WF 

0.0502 

58 

1270010453976WF 

0.8396 

661 001 37281 70WF 

10 

582101 077131 3WF 

0.0496 

59 

6615013528570WF 

0.7655 

662501 11 46771  WF 

10 

594501 1709363WF 

0.0443 

60 

6615013517337WF 

0.7344 

128001 2804855WF 

9 

6615011297445WF 

0.0342 

61 

661501 1496398WF 

0.6773 

129001 3041615WF 

9 

599801 3227746WF 

0.0243 

62 

598501 31 64589WF 

0.5741 

127001 1336494WF 

8 

129001 304161 5WF 

0.0234 

63 

662501 11 46771  WF 

0.2753 

598501 31 64589WF 

8 

661501 1496398WF 

0.0214 

64 

1290013041615WF 

0.2074 

6615013528570WF 

7 

661 001 039781 7WF 

0.0202 

65 

661 001 37281 70WF 

0.0959 

127001 0946872WF 

6 

6615010427834WF 

0.0158 

66 

594501 1709363WF 

0.059 

127001 0453976WF 

3 

662501 11 46771  WF 

0.0089 
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Criteria  Choose  an  item  with  high  activity 

-  High  expected  base  backorders 

-  A  requirement  greater  than  50  units 

-  A  demand  rate  greater  than  .5 

Ranks  =  (EBO  rank)  +  (REQ  rank)  +  (DR  rank) 
Choose  items  with  a  low  total  rank 

Meet  Criteria  Do  Not  Meet  Criteria 

NSN  Ranks  NSN  Ranks 

1 27001 3590877WF 
12700 12383662WF 
598501 21 22950WF 
12700 11 022965WF 
127001 1022966WF 
6625011 938861 WF 


Figure  A.1  Method  for  Choosing  an  Active  Item 


12700 13966750WF  9+3+3=15 
1 270992512706WF  7+6+ 1 3=26 
589501 1420803WF  4+14+10=28 
589501 248901 2WF  43+8+6=57 
66050 10876645WF  I  46+12+8=66j 
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AFMC/SAO  Data  Set 

There  are  three  data  sets  in  this  appendix:  scenario  data,  item  data,  requirements  data. 
AFMC/SAO  assembled  the  data  sets  in  order  to  conduct  an  assessment  of  the  effects  of  depot  stocking 
strategies  on  expected  base  backorders  (“Setting,”  1995:unnumbered).  The  data  sets  encompass  66  items 
(LRUs)  on  more  than  1,400  F-16/A,B,C,D  aircraft  stationed  at  55  installations  around  the  world.  They 
include  data  pertaining  to  flying  rates,  item  management,  and  requirements.  Descriptions  of  the  data  set 
terms  can  be  found  in  the  List  of  Terms.  The  data  categories  and  specific  fields  are  portrayed  in  Table 
B.l. 


Table  B.l  Data  Set  Descriptions 


Scenario  data:  Flying  hours  per  day  per  base  (FH) 

Item  data:  Demand  rate  per  100  flying  hours  (DR) 

Base  Processing  Percentage  (BP) 

Quantity  per  Application  (QPA) 

Application  Percentage  (AP) 

Requirements  data:  Order  and  Ship  Time  (OST) 

Base  Repair  and  Cycle  Time  (BRCT) 

Depot  Repair  Cycle  Time  (DRCT) 

OST  Pipeline  Quantity  (OSTQ) 

BRCT  Pipeline  Quantity  (BRCTQ) 

Base  Safety  Level  Quantity  (BSL) 

DRCT  Pipeline  Quantity  (DRCTQ) 

Requirements  (REQ) 

The  scenario  and  item  data  sets  come  from  AFMC  databases  which  collect  information  on  operations. 

The  same  holds  true  for  the  OST,  BRCT,  and  DRCT  fields  in  the  requirements  data  set.  The  D041 
produced  the  remaining  fields  of  the  requirements  data  set.  The  primary  purpose  of  the  D041  is  similar 
to  RBL.  The  D041  essentially  makes  a  prioritized  “shopping  list”  of  items  by  which  the  highest  priority 
item  will  make  the  largest  marginal  increase  in  aircraft  availability.  Given  a  requirement  of  a  certain 
item,  RBL  allocates  the  items  between  the  bases  and  the  depot  in  such  a  way  that  minimizes  the  number  of 
expected  base  backorders. 
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Substitute  Method  for  Providing  RBL  the  D041  Requirement  Input 

Requirement 

The  full  requirement  for  an  item  consists  of  pipeline  quantities  and  safety  levels.  The  requirement 
(REQ)  is  calculated  as  follows: 

REQ  =  BRCTQ  +  OSTQ  +  BSL  +  DRCTQ  +  DSL 

Pipeline  Equations 

The  pipeline  quantities  for  an  item  are  computed  in  a  straightforward  fashion,  summing  over  all 
bases  that  use  the  item: 

The  Base  Repair  Cycle  Pipeline  Quantity  (BRCTQ): 

BRCTQ  =  *QPAi  *APi  *BPi  *BRCTi 

i  =  1 

The  Order  and  Ship  Time  Pipeline  Quantity  (OSTQ): 

m 

OSTQ  =  *DRi  *QPAi  *APi  *(l-BPi)*OSTi 

i  =  1 

The  Depot  Repair  Cycle  Pipeline  Quantity  (DRCTQ): 

m 

DRCTQ  =  J^FH.  *DRi  *QPAi  *APi  *(l-BPi)*DRCTi 

i  =  1 

These  quantities  are  computed  for  each  item  (this  study  only  covered  one  item).  Incorporated  within  the 
pipeline  quantity  equations  are  the  following: 

The  Item  Flying  Hours  per  Day  (Item  FH): 

m 

ItemFH  = 


C-l 


The  Daily  Demand  Rate  (DDR): 


DDR  =  J^ItemFH;  *DRi 

i=  1 

Incorporating  base  processing  percentage  into  DDR  emphasizes  items  either  staying  at  the  base  or  going 
to  the  depot,  depending  on  the  pipeline  equation.  Finally,  incorporating  the  pipeline  time  results  in  the 
number  of  items  that  are  in  the  pipeline  at  any  point  in  time. 

Safety  Level  Regression 

The  base  and  depot  safety  levels  are  two  remaining  quantities  necessary  to  compute  a 
requirement.  They  attempt  to  account  for  variability  in  their  respective  pipelines  (see  List  of  Terms  for 
base  pipeline  and  depot  pipeline).  A  simple  way  to  account  for  these  two  quantities  was  to  develop  a 
regression  accounting  for  the  total  safety  level  (TSL)  as  a  function  of  the  pipelines: 

TSL{BSL+DSL}  =f(OSTQ,BRCTQ,DRCTQ) 

DRCTQ  was  not  significant  in  the  first  regression.  The  results  of  the  final  regression  are  included  in 
Figure  C.  1 .  The  regression  equation  used  for  determining  the  total  safety  level: 

TSL  =  13.572  +  1.023  OSTQ  +  1.866  BRCTQ  (C-1) 

There  is  another  type  of  stock  called  the  special  level  quantity  (see  List  of  Terms).  Due  to  its 
heavy  dependence  on  a  base’s  special  circumstances,  AFMC/SAO  suggested  it  not  be  included. 

The  impact  of  Equation  C-1  on  the  requirement  quantity  was  assessed.  If  Equation  S-l’s 
prediction  error  was  small  relative  to  the  requirement  quantity,  then  adverse  effects  on  the  requirement 
quantity  would  be  minimal.  However,  the  prediction  error  was  large.  Calculations  were  based  on  D041 
output  data  containing  pipeline  and  safety  level  information  for  the  66  items  shown  in  Appendix  A. 

Figure  C.2  plots  the  correlation  between  the  observed  total  safety  level  (BSL  +  DSL)  and  the  predicted 
safety  level  (Equation  3-1).  Because  the  plotted  points  are  not  tight  and  following  the  observed  values,  the 
plot  graphically  substantiates  the  low  R^  value  in  Figure  C.  1.  Table  C.  1  presents  statistics  on  the  impact 
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SUMMARY  OUTPUT 

Regression  Statistics 

Multiple  R 

0.76629 

R  Square 

0.58720 

Adjusted  R  Squar 

0.57409 

Standard  Error 

15.42067 

Observations 

66 

ANOVA 

df 

SS 

MS 

F 

Regression 

2 

21310.05851 

10655.03 

44.80724 

Residual 

63 

14981.21422 

237.7971 

Total 

65 

36291.27273 

— 

Coefficients 

Standard  Error 

tStat 

P-value 

Intercept 

13.57229 

2.60195 

5.21619 

2.16E-06 

OSTQ 

1.02303 

0.26860 

3.80873 

0.000319 

BRCTQ 

1.86593 

0.26250 

7.10828 

1.29E-09 

Figure  C.1  Safety  Level  Regression  Results 


Figure  C.2  Predicted  TSL  vs.  Observed  TSL 


of  Equation  B-l’s  predicted  safety  level  values.  The  errors  are  substantial,  and  because  the  TSL  can  make 
up  a  large  proportion,  if  not  all,  of  the  requirement,  the  large  prediction  errors  have  a  large  adverse  effect 
onREQ. 
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Table  Cl  Statistics  on  Impact  of  TSL  Prediction 


TSL  as  a  Percentage  of 
REQ  (Observed  values) 

TSL  Prediction  Error: 
Resid.  /  Observed  TSL 
(Absolute  Value:  %) 

TSL  Prediction  Error  as 
a  Percentage  of 
Observed  REQ 

Average 

57 

39.8 

43.0 

Minimum 

0 

1.1 

0.7 

Maximum 

100 

104 

579 
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Factor  Levels 

AFMC/SAO  suggested  all  of  the  factor  levels.  No  analysis  per  se  was  responsible  for  the  chosen 
levels.  The  ranges  over  which  the  factors  varied  come  from  a  combination  of  AFMC/SAO  logistic  analyst 
experience.  Air  Staff  goals,  and  existing  maximums  used  in  supply  models. 

The  variations  for  FH,  AP,  DR,  and  BP  are  based  on  experience.  The  low  levels  for  BRCT,  OST, 
and  DRCT  incorporate  Air  Staff  targets,  namely:  BRCT  (3),  OST  (1 1),  DRCT  (80%  of  current  value). 

For  the  NSN  used  in  this  study,  the  target  had  already  been  surpassed  for  OST,  so  the  current  level  served 
as  its  low  OST  level.  The  high  levels  for  BRCT  and  OST  incorporate  Standard  Base  Supply  System 
(SBSS)  caps,  or  limits.  The  limits  were  10  and  24  for  BRCT  and  OST,  respectively.  However,  since  the 
data  set  described  in  Appendix  B  used  integer  values  for  the  pipelines,  the  limits  were  reduced  by  one 
[BRCT  (9),  OST  (23)]  in  order  to  provide  an  integer  center  point  for  each  of  them.  The  high  level  for 
DRCT  was  chosen  to  be  the  same  percentage  deviation  from  the  eurrent  level  as  the  Air  Staff  target,  that 
is,  a  20%  increase.  The  Air  Staff  targets  and  SBSS  caps  apply  to  all  items,  not  just  the  one  used  in  this 
research. 
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First-Order  Design  and  Example  Input 


Table  E.  1  shows  the  64  coded  design  points  and  RBL  output  for  the  first-order  strategy. 


Table  £.1  First-Order  Design  and  RBL  Output 
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Table  E.1  (continued) 


FH 

AP 

DR 

BP 

BRCT  OST  DRCT 

REQ 

EBO 

1 

1 

1 

- 

1 

-1 

1 

1 

341 

14.6264 

-1 

-1 

-1 

1 

-1 

1 

1 

46 

4.2952 

1 

-1 

-1 

1 

-1 

1 

- 

1 

60 

5.6152 

-1 

1 

-1 

1 

-1 

1 

- 

1 

51 

4.8517 

1 

1 

-1 

1 

-1 

1 

1 

72 

6.3742 

-1 

-1 

1 

1 

-1 

1 

- 

1 

71 

6.3746 

1 

-1 

1 

1 

-1 

1 

1 

103 

8.2945 

-1 

1 

1 

1 

-1 

1 

1 

86 

1 

1 

1 

1 

-1 

1 

- 

1 

117 

WTBcB 

-1 

-1 

-1 

- 

1 

1 

1 

1 

138 

7.6271 

1 

-1 

-1 

- 

1 

1 

1 

- 

1 

184 

8.7465 

-1 

1 

-1 

1 

1 

1 

- 

1 

151 

1 

1 

-1 

- 

1 

1 

1 

239 

9.6065 

-1 

-1 

1 

- 

1 

1 

1 

- 

1 

224 

9.4337 

1 

-1 

1 

- 

1 

1 

1 

1 

360 

10.8426 

-1 

1 

1 

- 

1 

1 

1 

1 

293 

10.2733 

1 

1 

1 

- 

1 

1 

1 

- 

1 

395 

10.7383 

-1 

-1 

-1 

1 

1 

1 

- 

1 

88 

4.9195 

1 

-1 

-1 

1 

1 

1 

1 

127 

6.1358 

-1 

1 

-1 

1 

1 

1 

1 

105 

1 

1 

-1 

1 

1 

1 

- 

148 

6.4285 

-1 

-1 

1 

1 

1 

1 

1 

154 

6.5861 

1 

-1 

1 

1 

1 

1 

- 

221 

7.2424 

-1 

1 

1 

1 

1 

1 

- 

180 

6.8322 

1 

1 

1 

1 

1 

1 

1 

268 

7.5371 

Table  E.2  shows  the  first  design  point,  uncoded,  for  six  of  the  45  bases  that  use  the  item.  The 
settings  in  this  table  correspond  to  the  first  design  point  in  Table  E.  1.  “Users”  field  is  the  number  of 
bases  that  use  the  item,  “NCT”  is  the  condemnation  rate  which  was  not  considered  in  this  study.  “C- 
factor”  denotes  whether  the  base  is  situated  overseas  or  in  the  continental  United  States  and  also  was  not 
studied. 


Table  £.2  Partial  Uncoded  Design  Point  for  RBL  Input 


NSN  Users  DRCT  REQ 


1270013966750 

45 

30 

80 

Base 

DDR 

BP 

BRCT 

NCT 

OST 

C-factor 

FB2027 

0.21526 

0.59 

3.00 

0.00 

9 

1.00 

FB2805 

0.05118 

0.59 

3.00 

0.00 

9 

1.00 

FB2823 

0.04002 

0.59 

3.00 

0.00 

9 

1.00 

FB4488 

0.04511 

0.59 

3.00 

0.00 

9 

1.00 

FB4803 

0.12855 

0.59 

3.00 

0.00 

9 

1.00 

FB4823 

0.07495 

0.59 

3.00 

0.00 

9 

1.00 
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Experiment  Output  for  First-Order  Strategy 


Table  F.l  R£Q  and  EBO  Distribution  Statistics 


REQ 

EBO 

Mean 

152.9219 

8.1140 

Median 

141.0000 

7.5105 

Std.  Dev. 

81.2109 

2.4887 

Skewness 

0.9690 

0.5549 

Kurtosis 

0.6840 

-0.2415 

Table  F.2  EBO  Values  for  High  and  Low  Factor  Levels 


FH+ 

AP+ 

DR+ 

BP  + 

BRCT+ 

OST+ 

DRCT+ 

Mean 

8.8709 

8.4470 

9.2649 

6.2621 

7.7657 

8.4104 

8.2413 

Std.  Dev. 

2.5315 

2.5558 

2.4482 

1.2429 

1.8581 

2.5554 

2.5621 

FH- 

AP- 

DR- 

BP- 

BRCT- 

OST- 

DRCT- 

Mean 

7.3571 

7.7809 

6.9630 

9.9658 

8.4622 

7.8175 

7.9866 

Std.  Dev. 

2.2351 

2.4137 

1.9642 

1.9905 

2.9806 

2.4239 

2.4472 

Table  F.3  Correlation  Matrix 


FH 

AP 

DR 

BP 

BRCT 

OST 

DRCT 

REQ 

EBO 

FH 

1 

AP 

0 

1 

DR 

0 

0 

1 

BP 

0 

0 

0 

1 

BRCT 

0 

0 

0 

0 

1 

OST 

0 

0 

0 

0 

0 

1 

DRCT 

0 

0 

0 

0 

0 

0 

1 

REQ 

0.345 

0.1623 

0.5176 

-0.472 

0.4315 

0.2116 

0.0754 

1 

EBO 

0.3065 

0.1349 

0.4661 

-0.75 

-0.141 

0.1201 

0.0516 

0.706 

1 
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Figure  F.2  CDFofEBO 
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Figure  F.5  EBO  vs.  DR 
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Figure  F.9  EBO  vs.  DRCT 


Figure  F.IO  EBO  vs.  REQ 
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SAS  Code  For  First-Order  Strategy 


OPTIONS  NODATE  NONUMBER  LINESIZE=  80; 

TITLE  'Thesis:Applying  RSM  to  RBL  by  Captain  Todd  May'; 

'nTLE2  "First-Order  Strategy'; 

DATA  CUBIC; 

INFILE  FACTORAL; 

INPUT  FH  AP  DR  BP  BRCT  OST  DRCT  REQ  EBO; 

PROC  PLOT  DATA=CUBIC; 

PLOT  EBO*FH;  PLOT  EBO*AP;  PLOT  EBO*DR;  PLOT  EBO*BP;  PLOT  EBO*BRCT; 
PLOT  EBO*OST;  PLOT  EBO*DRCT;  PLOT  EBO*REQ;  PLOT  REQ*FH;  PLOT  REQ*AP; 
PLOT  REQ*DR;  PLOT  REQ*BP;  PLOT  REQ*BRCT;  PLOT  REQ*OST;  PLOT  REQ*DRCT; 
DATA  NORM64  /*  Normal  prob.  points  for  QQ  plots  */; 

DO  1=1  TO  64; 

J=(I-.5)/64; 

OUTPUT; 

END; 

PROC  SORT  DATA=CUBIC  OUT=SORTCUBE; 

BY  EBO; 

DATA  QQEBO  /*  Data  set  of  EBO  and  QQ  points  */; 

SET  NORM64  (KEEP=J); 

SET  SORTCUBE  (KEEP=EBO); 

PROC  PLOT  D  ATA=QQEBO  /*  QQ  plot  of  EBO  */; 

PLOT  J*EBO='*'; 

T1TLE3  'QQ  Plot  of  EBO'; 

PROC  SORT  DATA=CUBIC  OUT=SORTREQ; 

BY  REQ; 

DATA  QQREQ  /*  Data  set  of  REQ  and  QQ  points  */; 

SET  NORM64  (KEEP=J); 

SET  SORTREQ  (KEEP=REQ); 

PROC  PLOT  D  ATA=QQREQ  /*  QQ  plot  of  REQ  */; 

PLOT  J*REQ='*'; 

TITLES  'QQ  Plot  of  REQ'; 

TITLES; 

PROC  RSQUARE  DATA=CUBIC  /*  Find  models  with  highest  R-square  */ 

OUTEST=EST  ADJRSQ  SSE  MSE  CP  B  SELECT=1; 

MODEL  EBO=FH  AP  DR  BP  BRCT  OST  DRCT; 

PROC  PRINT  DATA=EST; 

PROC  PLOT  DATA=EST; 

PLOT  _CP_*_P_='C'  _P_*_P_='P'  /  OVERLAY; 

TITLES  'Best  First-Order  Models  Without  REQ'; 

TITLES; 

PROC  RSQUARE  DATA=CUBIC  /*  Same  procedure  including  REQ  */ 

OUTEST=EST  ADJRSQ  SSE  MSE  CP  B  SELECT=1; 

MODEL  EBO=FH  AP  DR  BP  BRCT  OST  DRCT  REQ; 

PROC  PRINT  DATA=EST; 

PROC  PLOT  DATA=EST; 

PLOT  _CP_*_P_='C'  _P_*_P_=’P'/OVERLAY; 
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TITLES  'Best  First-Order  Models  with  REQ'; 

TITLES; 

DATA  EFFECT; 

INFILE  EFFECTS; 

INPUT  LABEL  $  VALUE; 

VALUE=VALUE*2; 

PROC  SORT  DATA=EFFECT; 

BY  VALUE; 

DATA  NORM?; 

DO  1=1  TO  7; 

J=(I-.5)/7; 

OUTPUT; 

END; 

DATA  PLOTl  /*  Factor  effects  (except  REQ)  and  QQ  points  */; 

SET  NORM?; 

SET  EFFECT; 

PROC  PLOT  DATA=PLOTl  /*  QQ  plot  of  factor  effects  (except  REQ)  */; 
PLOT  J*VALUE=LABEL; 

TITLES  'QQ  Plot  of  Factor  Effects'; 

TITLES; 

PROC  STEPWISE  DATA=CUBIC; 

MODEL  EBO=FH  AP  DR  BP  BRCT  OST  DRCT  REQ/MAXR; 
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Appendix  H 

SAS  RSQUARE  Output  For  First-Order  Strategy 

Thesis:  Applying  RSM  to  RBL  by  Captain  Todd  May 
First-Order  Strategy 

N  =  64  Regression  Models  for  Dependent  Variable:  EBO 


Parameter 

Rsq  Adj  C(p)  MSE  SSE  Estimates 
In  Rsq  Intercept  FH  AP  DR 

BP  BRCT  OST  DRCT  REQ 


1 

0.5625 

0.5554 

286.8  2.753  170.7  8.1140 

-1.8519 

2 

0.7798 

0.7726 

116.6 

1.409  85.9  8.1140 

.  1.1509 

-1.8519 

3 

0.8737 

0.8674  44.08  0.821  49.3  8.1140 

0.7569 

.  1.1509 

-1.8519 

4 

0.8936 

0.8864 

30.32 

0.703  41.5  8.1140 

0.7569 

.  1.1509 

-1.8519  -0.3482 

5 

0.9149 

0.9076 

15.43 

0.572  33.2  6.6010 

0.4819 

.  0.7384 

-1.4758  -0.6922 

.  0.00989 

6 

0.9262 

0.9185 

8.467 

0.505  28.8  8.1140 

0.7569 

0.3331  1.1509 

-1.8519  -0.3482  0.2965 

7 

0.9289 

0.9200 

8.358 

0.495  27.7  8.1140 

0.7569 

0.3331  1.1509 

-1.8519  -0.3482  0.2965  0.1274 

8 

0.9306 

0.9205 

9.000 

0.492  27.1  7.5709 

0.6582 

0.2866  1.0028 

-1.7169  -0.4717  0.2359  0.1058  0.00355 
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Figure  LI  Interaction  Between  FH  and  AP 


Figure  L3  Interaction  Between  FH  and  BP 


12 

11 


( 


-2-1012 

FH  level 


Figure  L4  Interaction  Between  FH  and  OST 


Figure  L5  Interaction  Between  FH  and  DRCT 


Figure  L6  Interaction  Between  AP  and  DR 
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Figure  1.9  Interaction  Between  AP  and  OST 
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Figure  LIO  Interaction  Between  AP  and  DRCT 


Figure  Lll  Interaction  Between  DR  and  BP 


Figure  1.12  Interaction  Between  DR  and  BRCT 


Figure  L13  Interaction  Between  DR  and  OST 


Figure  1.14  Interaction  Between  DR  and  DRCT 
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Figure  1.15  Interaction  Between  BP  and  BRCT 


Figure  1.16  Interaction  Between  BP  and  OST 


Figure  1.17  Interaction  Between  BP  and  DRCT 


12 

11 
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Figure  L18  Interaction  Between  BRCT  and  OST 


Figure  L19  Interaction  Between  BRCT  and  DRCT 


12 
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Figure  L20  Interaction  Between  OST  and  DRCT 
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Center  and  Axial  Design  Points  for  Second-Order  Design 


Table  J.l  Coded  Center  and  Axial  Points  for  Second-Order  Design 


1  FH 

AP 

DR 

BP 

BRCT 

OST 

DRCT 

REQ 

EBO  1 

■■ 

0 

0 

0 

0 

154 

-1 

0 

0 

0 

0 

126 

7.9682 

1 

0 

0 

0 

0 

182 

9.6661 

0 

-1 

0 

0 

0 

8.4965 

0 

1 

0 

0 

166 

0 

-1 

0 

0 

112 

0 

1 

0 

0 

196 

9.9338 

0 

-1 

0 

0 

192 

0 

1 

0 

116 

6.8452 

0 

-1 

0 

119 

0 

1 

0 

189 

0 

-1 

0 

137 

8.3493 

0 

0 

0 

1 

0 

171 

9.3123 

0 

-1 

148 

0 

1 

160 

9.14261 
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Experiment  Output  Information  for  Second-Order  Strategy 


Table  K.1  R£Q  and  EBO  Distribution  Statistics 


REQ 

EBO 

Mean 

153.1013 

8.2503 

Median 

145.0000 

8.2022 

Std.  Dev. 

73.9831 

2.2926 

Skewness 

1.0226 

0.4207 

Kurtosis 

1.2899 

-0.0113 

Table  IC2  EBO  Values  for  Center  Levels 


FH 

AP 

DR 

BP 

BRCT 

OST 

DRCT 

Mean 

9.5706 

9.5567 

9.5936 

9.5821 

9.5762 

9.5683 

9.5427 

Std.  dev. 

1.0023 

1.0472 

0.9262 

0.7209 

1 .0334 

1.0421 

1.0564 

Table  K3  Correlation  Matrix 


FH 

AP 

DR 

BP 

BRCT 

OST 

DRCT 

REQ 

EBO 

FH 

1 

AP 

0 

1 

DR 

0 

0 

1 

BP 

0 

0 

0 

1 

BRCT 

0 

0 

0 

0 

1 

OST 

0 

0 

0 

0 

0 

1 

DRCT 

0 

0 

0 

0 

0 

0 

1 

REQ 

0.3457 

0.1626 

0.5186 

-0.4727 

0.4323 

0.2119 

0.0755 

1 

EBO 

0.3048 

0.1345 

0.4631 

-0.7437 

-0.1426 

0.1212 

0.0515 

0.7017 

1 

K-1 


Figure  K.2  CDFofEBO 
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Figure  K.5  EBO  vs.  DR 


Figure  K.6  EBO  vs.  BP 
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Figure  K.7  EBO  vs.  BRCT 


Figure  K.8  EBO  vs.  OST 
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Figure  K.9  EBO  vs.  DRCT 


Figure  K.10  EBO  vs.  REQ 
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Appendix  L 

SAS  Code  for  Second-Order  Strategy 


OPTIONS  NODATE  NONUMBER  LINESIZE=80; 

TITLE  'Thesis;  Applying  RSM  to  RBL  by  Captain  Todd  May'; 

TITLE2  'Second-Order  Strategy'; 

DATA  CUBIC; 

INFILE  FACTORAL;  INPUT  FH  AP  DR  BP  BRCT  OST  DRCT  REQ  EBO; 
DATA  CENTER22; 

INFILE  CENTER22;  INPUT  FH  AP  DR  BP  BRCT  OST  DRCT  REQ  EBO; 
DATA  CENTER; 

INFILE  CENTER;  INPUT  FH  AP  DR  BP  BRCT  OST  DRCT  REQ  EBO; 
DATA  AXIAL; 

INFILE  AXIAL;  INPUT  FH  AP  DR  BP  BRCT  OST  DRCT  REQ  EBO; 
DATA  ALL22; 

SET  CUBIC  CENTER22  AXIAL; 

DATAALLl; 

SET  CUBIC  CENTER  AXIAL; 

PROC  RSREG  DATA=ALL22; 

TITLES  'With  22  Center  Points  |  Without  REQ'; 

MODEL  EBO=FH  AP  DR  BP  BRCT  OST  DRCT; 

PROC  RSREG  DATA=ALL1; 

TITLES  'With  1  Center  Point  |  Without  REQ'; 

MODEL  EBO=FH  AP  DR  BP  BRCT  OST  DRCT; 

PROC  RSREG  DATA=ALL22; 

TITLES  'With  22  Center  Points  |  With  REQ'; 

MODEL  EBO=FH  AP  DR  BP  BRCT  OST  DRCT  REQ; 

PROC  RSREG  DATA=ALL1; 

TITLES  'With  1  Center  Point  |  With  REQ'; 

MODEL  EBO=FH  AP  DR  BP  BRCT  OST  DRCT  REQ; 

DATA  NORM?; 

DO  1=1  TO  7;  J=(I-.5)/7;  OUTPUT;  END; 

DATANORM21; 

DO  1=1  TO  21;  J=(l-.5)/21;  OUTPUT;  END; 

DATA  ONEWAY; 

INFILE  ONEWAY;  INPUT  LABEL  $  VALUE; 

PROC  SORT  DATA=ONEWAY; 

BY  VALUE; 

DATA  TWOWAY; 

INFILE  TWOWAY;  INPUT  LABEL  $  VALUE; 

PROC  SORT  DATA=TWOWAY; 

BY  VALUE; 

DATA  QQONEWAY; 

SET  NORM?  (KEEP=J);  SET  ONEWAY; 

DATA  QQTWOWAY; 

SETNORM21  (KEEP=J);  SET  TWOWAY; 

PROC  PLOT  DATA=QQONEWAY; 

TITLES  'QQ  Plot  of  One-Way  Parameter  Estimates  w/o  REQ'; 

PLOT  J*VALUE=LABEL; 
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PROC  PLOT  DATA=QQTWOWAY; 

TITLES  'QQ  Plot  of  Two-Way  Parameter  Estimates  w/o  REQ'; 

PLOT  J*VALUE=LABEL; 

PROC  MEANS  DATA=ALL1  NWAY  NOPRINT;  CLASS  REQ  FH;  VAR  EBO; 

OUTPUT  OUT=REQFH  MEAN=; 

PROC  MEANS  DATA=ALL1  NWAY  NOPRINT;  CLASS  REQ  AP;  VAR  EBO; 

OUTPUT  OUT=REQAP  MEAN==; 

PROC  MEANS  DATA=ALL1  NWAY  NOPRINT;  CLASS  REQ  DR;  VAR  EBO; 

OUTPUT  OUT=REQDR  MEAN=; 

PROC  MEANS  DATA=ALL1  NWAY  NOPRINT;  CLASS  REQ  BP;  VAR  EBO; 

OUTPUT  OUT=REQBP  MEAN=; 

PROC  MEANS  DATA=ALL1  NWAY  NOPRINT;  CLASS  REQ  BRCT;  VAR  EBO; 
OUTPUT  OUT=REQBRCT  MEAN=; 

PROC  MEANS  DATA=ALL1  NWAY  NOPRINT;  CLASS  REQ  OST;  VAR  EBO; 

OUTPUT  OUT=REQOST  MEAN=; 

PROC  MEANS  DATA=ALL1  NWAY  NOPRINT;  CLASS  REQ  DRCT;  VAR  EBO; 

OUTPUT  OUT=REQDRCT  MEAN=; 

TITLES  'Interaction  Plot'; 

PROC  PLOT  DATA=REQFH;  PLOT  EBO*FH=REQ; 

PROC  PLOT  DATA=REQAP;  PLOT  EBO*AP=REQ; 

PROC  PLOT  DATA=REQDR;  PLOT  EBO*DR=REQ; 

PROC  PLOT  DATA=REQBP;  PLOT  EBO*BP=REQ; 

PROC  PLOT  DATA=REQBRCT;  PLOT  EBO*BRCT=REQ; 

PROC  PLOT  DATA=REQOST;  PLOT  EBO*OST=REQ; 

PROC  PLOT  DATA=REQDRCT;  PLOT  EBO*DRCT=REQ; 

DATA  ALLSTEP; 

SET  ALLl;  A=FH;  B=AP;  C=DR;  D=BP;  E=BRCT;  F=OST;  G=DRCT;  H=REQ; 
AA=A*A;  BB=B*B;  CC=C*C;  DD=D*D;  EE=E*E;  FF=F*F;  GG=G*G;  HH=H*H; 
AB=A*B;  AC=A*C;  AD=A*D;  AE=A*E;  AF=A*F;  AG=A*G;  AH=A*H;  BC=B*C; 
BD=B*D;  BE=B*E;  BF=B*F;  BG=B*G;  BH=B*H;  CD=C*D;  CE=C*E;  CF=C*F; 
CG=C*G;  CH=C*H;  DE=D*E;  DF=D*F;  DG=D*G;  DH=D*H;  EF=E*F;  EG=E*G; 
EH=E*H;  FG=F*G;  FH=F*H;  GH=G*H; 

TITLES; 

PROC  RSQUARE  DATA=ALLSTEP  OUTEST=EST  ADJRSQ  SSE  MSE  CP  B  SELECT=1; 
MODEL  EBO=  A  B  C  D  E  F  G  H  AA  BB  CC  DD  EE  FF  GG  HH  AB  AC  AD  AE 
AF  AG  AH  BC  BD  BE  BF  BG  BH  CD  CE  CF  CG  CH  DE  DF  DG  DH  EF 
EGEHFGFHGH; 

PROC  STEPWISE  DATA=ALLSTEP; 

MODEL  EBO=  ABCDEFGHAABBCCDDEEFFGGHHABACADAE 
AF  AG  AH  BC  BD  BE  BF  BG  BH  CD  CE  CF  CG  CH  DE  DF  DG  DH  EF 
EG  EH  FG  FH  GH/MAXR; 
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Appendix  M 

SAS  RSQUARE  Output  For  Second-Order  Strategy 

Thesis:Applying  RSM  to  RBL  by  Captain  Todd  May 
Second-Order  Strategy 

N  =  79  Regression  Models  for  Dependent  Variable:  EBO 


Parameter 

Rsq  Adj  C(p)  MSE  SSE  Estimates 
In  Rsq  Intercept  ABC 


D 

E 

F 

G  H 

AA 

BB 

CC 

DD 

EE 

FF 

GG 

HH 

AB 

AC 

AD 

AE 

AF 

AG 

AH 

BC 

BD 

BE 

BF 

BG 

BH 

CD 

CE 

CF 

CG 

CH 

DE 

DF 

DG 

DH 

EF 

EG 

EH 

FG 

FH 

GH 

1  0.5530  0.5472  12846  2.380  183.2  8.2503 
-1.8534  .... 


2  0.8384  0.8341 


4599  0.872  66 


3.8438 

0.0306 


-0.00962 


3  0.9254  0.9224  2085  0.408  30.6  4.8183 
-0.8805  .  0.0239 


-0.00799 
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Parameter 


Rsq 

Adj 

C(p) 

MSE 

SSE 

Estimates 

In 

Rsq 

Intercept 

A 

D 

E 

F 

G  H 

AA 

BB 

CC 

DD 

EE 

FF 

GG 

HH 

AB 

AC 

AD 

AE 

AF 

AG 

AH 

BC 

BD 

BE 

BF 

BG 

BH 

CD 

CE 

CF 

CG 

CH 

DE 

DF 

DG 

DH 

EF 

EG 

EH 

FG 

FH 

GH 

4  0.9607 

0.9586  1068 

0.218 

16.1 

2.5350 

.  0.0522 

-0.00008 


.  -0.00528 

.  -0.00773 


5  0.9726  0.9708  724.0  0.154  11.2  1.2633 

.  -1.3488  .  .  0.0651 

-0.00011  .... 

.  -0.3106 

.  -0.00517 


6  0.9806  0.9789  496.8  0.111  8.0  3.5063  0.3256  .  0.4502 

.  0.0457 

-0.00008  . 


.  -0.00697 

.  -0.00636 


7  0.9850  0.9835  370.1  0.087  6.1  1.3297 

.  -1.3725  .  0.0633 

-0.0001  .  -0.2104 

.  -0.3370 

.  0.2067  .-0.00495 
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Adj  C{p)  MSB  SSE  Estimates 
Rsq  Intercept  ABC 

D  E  F  G  H  AA 

BB  CC  DD  EE  FF  GG 

HH  AB  AC  AD  AE  AF 
AG  AH  BC  BD  BE  BF 

BG  BH  CD  CE  CF  CG 

CH  DE  DF  DG  DH  EF 

EG  EH  FG  FH  GH 

8  0.9884  0.9871  274.2  0.068  4.8  1.3389 

.  -1.3950  .  0.0625 

-0.0001  .  .  -0.2229 

.  -0.3548 

.  0.1851  .-0.00486 

.  -0.00094 


Parameter 

Rsq 

In 


9  0.9911  0.9899  199.2  0.053  3.7 

.  -1.3801 

.  -0.3329 

-0.00009 


1.7785 
.  0.0605 

.  -0.2290 


.  -0.3628 

.  0.1888  .-0.00494 

.  -0.00096 
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Appendix  N 

SAS  RSREG  Output  for  Second-Order  Strategy 


Thesis:  Applying  RSM  to  RBL  by  Captain  Todd  May 
Second-Order  Strategy 
With  1  Center  Point  |  With  REQ 

Response  Surface  for  Variable  EBO 

Response  Mean  8.250330 

RootMSE  0.119082 

R-Square  0.9988 

Coef  of  Variation  1.4434 


Degrees 

of  Type  I  Sum 

Regression  Freedom  of  Squares  R-Square  F-Ratio  Prob  >  F 

Linear  8  376.299742  0.9179  3317.0  0.0000 

Quadratic  8  16.234916  0.0396  143.1  0.0000 

Crossproduct  28  16.938236  0.0413  42.659  0.0000 

Total  Regress  44  409.472894  0.9988  656.3  0.0000 


Degrees 

of  Sum  of 

Residual  Freedom  Squares  Mean  Square 
Total  Error  34  0.482140  0.014181 

Degrees 

of  Parameter  Standard  T  for  HO: 

Parameter  Freedom  Estimate  Error  Parameter=0  Prob  >  |T| 


INTERCEPT 

1 

9.835604 

1.179608 

8.338 

0.0000 

FH 

1 

0.913769 

0.236621 

3.862 

0.0005 

AP 

1 

0.432539 

0.132931 

3.254 

0.0026 

DR 

1 

1.304706 

0.331180 

3.940 

0.0004 

BP 

1 

-1.892005 

0.443462 

-4.266 

0.0001 

BRCT 

1 

0.435110 

0.335136 

1.298 

0.2029 

OST 

1 

0.599052 

0.172097 

3.481 

0.0014 

DRCT 

1 

0.116336 

0.092599 

1.256 

0.2176 

REQ 

1 

-0.002073 

0.009899 

-0.209 

0.8354 

FH*FH 

1 

-0.127816 

0.084431 

-1.514 

0.1393 

AP*FH 

1 

0.033818 

0.026124 

1,295 

0,2042 

AP*AP 

1 

-0.044994 

0.079160 

-0.568 

0.5735 

DR*FH 

1 

0.055962 

0.058758 

0.952 

0.3476 
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Degrees 

of  Parameter  Standard  T  for  HO: 

Parameter  Freedom  Estimate  Enor  Parameter=0  Prob  >  |T| 


DR*AP  1 

DR*DR  1 

BP*FH  1 

BP*AP  1 

BP*DR  1 

BP*BP  1 

BRCT*FH  1 

BRCT*AP  -1 

BRCT*DR  1 

BRCT*BP  1 

BRCT*BRCT  1 

OST*FH  1 

OST*AP  1 

OST*DR  1 

OST*BP  1 

OST*BRCT  1 

OST*OST  1 

DRCT*FH  1 

DRCT*AP  1 

DRCT*DR  1 

DRCT*BP  1 

DRCT*BRCT  1 

DRCT=»OST  1 

DRCT^DRCT  1 

REQ*FH  1 

REQ*AP  1 

REQ*DR  1 

REQ*BP  1 

REQ*BRCT  1 

REQ*OST  1 

REQ*DRCT  1 

REQ*REQ  1 


0.027548 

0.035098 

0.785 

0.4380 

-0.298809 

0.111825 

-2.672 

0.0115 

-0.167288 

0.072223 

-2.316 

0.0267 

-0.091889 

0.039971 

-2.299 

0.0278 

-0.241237 

0.104795 

-2.302 

0.0276 

-0.230176 

0.097093 

-2.371 

0.0236 

-0.035559 

0.057387 

-0.620 

0.5396 

-0.020199 

0.035270 

-0.573 

0.5706 

-0.070601 

0.080126 

-0.881 

0.3844 

0.356599 

0.076028 

4.690 

0.0000 

-0.018368 

0.083969 

-0.219 

0.8282 

0.029283 

0.029782 

0.983 

0.3324 

0.020075 

0.021154 

0.949 

0.3493 

0.039083 

0.039069 

1.000 

0.3242 

-0.269582 

0.088677 

-3.040 

0.0045 

-0.051798 

0.040898 

-1.267 

0.2139 

-0.083621 

0.079153 

-1.056 

0.2982 

0.001648 

0.021865 

0.0754 

0.9403 

-0.024601 

0.017310 

-1.421 

0.1644 

0.022658 

0.027609 

0.821 

0.4176 

-0.099056 

0.039345 

-2.518 

0.0167 

-0.010651 

0.025838 

-0.412 

0.6828 

-0.013700 

0.018284 

-0.749 

0.4588 

0.046734 

0.078307 

0.597 

0.5546 

0.000995 

0.001650 

0.603 

0.5504 

0.000313 

0.000869 

0.361 

0.7205 

0.002021 

0.002565 

0.788 

0.4361 

-0.002038 

0.002330 

-0.875 

0.3879 

-0.002984 

0.001568 

-1.903 

0.0656 

-0.000865 

0.001013 

-0.854 

0.3992 

0.000457 

0.000556 

0.823 

0.4162 

-0.000024 

0.000023 

-1.049 

0.3018 
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Appendix  O 

Canonical  Analysis 


Canonical  analysis  was  performed  on  the  44-term,  second-order  model  shown  in  Appendix  N, 
henceforth  referred  to  as  the  full  model.  Thus,  the  analysis  begins  with  a  second-order  model 
incorporating  all  main  effects,  two-way  interactions,  and  quadratic  terms.  The  canonical  form  A  rotates 
the  factor  axes  so  that  they  align  with  the  principal  axes  of  the  response  surface.  The  two-way  interactions 
are  removed  as  a  result  of  the  axis  rotation.  The  canonical  form  A  for  the  full  model  is: 

EBO  =  9.8356  +  0.5046  FH  -  0.3908  AP  +  0.7903  DR  -  0.9595  BP  +  0.5559  BRCT  + 

2.0464  OST  +  0.6450  DRCT  -  0.0117  REQ  +  0.2839  FH^  +  0.0473  AP^  -  0.0411  DR^  - 
0.0592  BP^  -  0.1276  BRCT^  -  0.2647  OST^  -  0.4368  DRCT^  -  0.9197  REQ^ 

The  canonical  form  B  offers  a  further  simplification  to  the  response  surface  representation.  If  the 
response  surface  has  a  stationary  point  within  the  design  region,  the  rotated  factor  axes  can  be  translated 
to  coincide  with  the  stationary  point.  The  translation  eliminates  the  linear  terms  from  the  model,  leaving 
only  the  constant  and  quadratic  terms.  For  the  response  surface  under  investigation,  the  canonical  form  B 
was  not  developed  because  a  stationary  point  did  not  exist. 
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Appendix  P 

Verification  Output 


Table  P.l  Verification  Design  and  Output  for  Factorial  Points 


FH 

AP 

DR 

BP 

BRCT 

OST 

DRCT 

REQ 

EBO  1 

-0.9 

-0.5 

-0.5 

-0.56 

-0.33 

-0.86 

0.8 

105 

0.5 

-0.5 

-0.5 

-0.56 

-0.33 

-0.86 

-0.4 

126 

8.8702 

-0.9 

0.5 

-0.5 

-0.56 

-0.33 

-0.86 

-0.4 

106 

7.8944 

0.5 

0.5 

-0.5 

-0.56 

-0.33 

-0.86 

0.8 

147 

9.6303 

-0.9 

-0.5 

0.5 

-0.56 

-0.33 

-0.86 

-0.4 

127 

8.9122 

0.5 

-0.5 

0.5 

-0.56 

-0.33 

-0.86 

0.8 

178 

10.7262 

-0.9 

0.5 

0.5 

-0.56 

-0.33 

-0.86 

0.8 

149 

9.5546 

0.5 

0.5 

0.5 

-0.56 

-0.33 

-0.86 

-0.4 

181 

10.8817 

-0.9 

-0.5 

-0.5 

0.44 

-0.33 

-0.86 

-0.4 

79 

6.1665 

0.5 

-0.5 

-0.5 

0.44 

-0.33 

-0.86 

0.8 

106 

7.3630 

-0.9 

0.5 

-0.5 

0.44 

-0.33 

-0.86 

0.8 

89 

6.6184. 

0.5 

0.5 

-0.5 

0.44 

-0.33 

-0.86 

-0.4 

110 

7.6519 

-0.9 

-0.5 

0.5 

0.44 

-0.33 

-0.86 

0.8 

107 

7.3779 

0.5 

-0.5 

0.5 

0.44 

-0.33 

-0.86 

-0.4 

133 

8.4800 

-0.9 

0.5 

0.5 

0.44 

-0.33 

-0.86 

-0.4 

111 

7.6783 

0.5 

0.5 

0.5 

0.44 

-0.33 

-0.86 

0.8 

151 

9.0671 

-0.9 

-0.5 

-0.5 

-0.56 

0.67 

-0.86 

-0.4 

118 

7.2865 

0.5 

-0.5 

-0.5 

-0.56 

0.67 

-0.86 

0.8 

163 

8.6734 

-0.9 

0.5 

-0.5 

-0.56 

0.67 

-0.86 

0.8 

136 

7.8105 

0.5 

0.5 

-0.5 

-0.56 

0.67 

-0.86 

-0.4 

167 

8.9019 

-0.9 

-0.5 

0.5 

-0.56 

0.67 

-0.86 

0.8 

164 

8.7490 

0.5 

-0.5 

0.5 

-0.56 

0.67 

-0.86 

-0.4 

202 

9.6534 

-0.9 

0.5 

0.5 

-0.56 

0.67 

-0.86 

-0.4 

168 

8.9675 

0.5 

0.5 

0.5 

-0.56 

0.67 

-0.86 

0.8 

235 

10.0898 

-0.9 

-0.5 

-0.5 

0.44 

0.67 

-0.86 

0.8 

108 

6.2707 

0.5 

-0.5 

-0.5 

0.44 

0.67 

-0.86 

-0.4 

136 

7.1925 

-0.9 

0.5 

-0.5 

0.44 

0.67 

-0.86 

-0.4 

114 

6.4498 

0.5 

0.5 

-0.5 

0.44 

0.67 

-0.86 

0.8 

153 

7.5389 

-0.9 

-0.5 

0.5 

0.44 

0.67 

-0.86 

-0.4 

137 

7.2248 

0.5 

-0.5 

0.5 

0.44 

0.67 

-0.86 

0.8 

185 

8.0964 

-0.9 

0.5 

0.5 

0.44 

0.67 

-0.86 

0.8 

154 

7.5808 

0.5 

0.5 

0.5 

0.44 

0.67 

-0.86 

-0.4 

195 

8.2407 

-0.9 

-0.5 

-0.5 

-0.56 

-0.33 

0.43 

-0.4 

119 

8.2417 

0.5 

-0.5 

-0.5 

-0.56 

-0.33 

0.43 

0.8 

164 

9.9101 

-0.9 

0.5 

-0.5 

-0.56 

-0.33 

0.43 

0.8 

137 

8.8439 

0.5 

0.5 

-0.5 

-0.56 

-0.33 

0.43 

-0.4 

168 

10.1980 

-0.9 

-0.5 

0.5 

-0.56 

-0.33 

0.43 

0.8 

165 

9.9935 

0.5 

-0.5 

0.5 

-0.56 

-0.33 

0.43 

-0.4 

204 

11.0578 

-0.9 

0.5 

0.5 

-0.56 

-0.33 

0.43 

-0.4 

170 

10.1437 

0.5 

0.5 

0.5 

-0.56 

-0.33 

0.43 

0.8 

236 

11.7272 

-0.9 

-0.5 

-0.5 

0.44 

-0.33 

0.43 

0.8 

92 

6.7732 

0.5 

-0.5 

-0.5 

0.44 

-0.33 

0.43 

-0.4 

115 

7.7659 

P-1 


Table  P.l  (continued) 


FH 

AP 

DR 

BP 

BRCT 

OST 

DRCT 

REQ 

EBO 

-0.9 

0.5 

-0.5 

0.44 

-0.33 

0.43 

-0.4 

96 

7.0258 

0.5 

0.5 

-0.5 

0.44 

-0.33 

0.43 

0.8 

129 

8.3431 

-0.9 

-0.5 

0.5 

0.44 

-0.33 

0.43 

-0.4 

115 

7.9012 

0.5 

-0.5 

0.5 

0.44 

-0.33 

0.43 

0.8 

156 

9.2741 

-0.9 

0.5 

0.5 

0.44 

-0.33 

0.43 

0.8 

130 

8.3793 

0.5 

0.5 

0.5 

0.44 

-0.33 

0.43 

-0.4 

163 

9.5498 

-0.9 

-0.5 

-0.5 

-0.56 

0.67 

0.43 

0.8 

146 

8.0685 

0.5 

-0.5 

-0.5 

-0.56 

0.67 

0.43 

-0.4 

182 

8.9332 

-0.9 

0.5 

-0.5 

-0.56 

0.67 

0.43 

-0.4 

151 

8.3340 

0.5 

0.5 

-0.5 

-0.56 

0.67 

0.43 

0.8 

208 

9.4792 

-0.9 

-0.5 

0.5 

-0.56 

0.67 

0.43 

-0.4 

183 

9.0116 

0.5 

-0.5 

0.5 

-0.56 

0.67 

0.43 

0.8 

253 

10.1873 

-0.9 

0.5 

0.5 

-0.56 

0.67 

0.43 

0.8 

210 

9.4565 

0.5 

0.5 

0.5 

-0.56 

0.67 

0.43 

-0.4 

263 

10.2523 

-0.9 

-0.5 

-0.5 

0.44 

0.67 

0.43 

-0.4 

115 

6.4554 

0.5 

-0.5 

-0.5 

0.44 

0.67 

0.43 

0.8 

153 

7.6083 

-0.9 

0.5 

-0.5 

0.44 

0.67 

0.43 

0.8 

128 

6.9296 

0.5 

0.5 

-0.5 

0.44 

0.67 

0.43 

-0.4 

162 

7.7356 

-0.9 

-0.5 

0.5 

0.44 

0.67 

0.43 

0.8 

154 

7.6489 

0.5 

-0.5 

0.5 

0.44 

0.67 

0.43 

-0.4 

197 

8.1896 

-0.9 

0.5 

0.5 

0.44 

0.67 

0.43 

-0.4 

163 

7.7812 

0.5 

0.5 

0.5 

0.44 

0.67 

0.43 

0.8 

221 

8.6070 

Table  P.2  Verification  Design  and  Output  for  Center  and  Axial  Points 


FH 

AP 

DR 

BP 

BRCT 

OST 

DRCT 

REQ 

EBO 

-0.25 

0 

0.17 

-0.11 

0 

-0.43 

0.4 

152 

9.0360 

-0.9 

0 

0.17 

-0.11 

0 

-0.43 

0.4 

133 

8.3779 

0.5 

0 

0.17 

-0.11 

0 

-0.43 

0.4 

174 

9.6853 

-0.25 

-0.5 

0.17 

-0.11 

0 

-0.43 

0.4 

146 

8.7100 

-0.25 

0.5 

0.17 

-0.11 

0 

-0.43 

0.4 

158 

9.2584 

-0.25 

0 

-0.5 

-0.11 

0 

-0.43 

0.4 

126 

8.0648 

-0.25 

0 

0.5 

-0.11 

0 

-0.43 

0.4 

166 

9.3829 

-0.25 

0 

0.17 

-0.56 

0 

-0.43 

0.4 

168 

9.7455 

-0.25 

0 

0.17 

0.44 

0 

-0.43 

0.4 

133 

8.0150 

-0.25 

0 

0.17 

-0.11 

-0.33 

-0.43 

0.4 

141 

9.1217 

-0.25 

0 

0.17 

-0.11 

0.67 

-0.43 

0.4 

175 

8.6184 

-0.25 

0 

0.17 

-0.11 

0 

-0.86 

0.4 

144 

8.7755 

-0.25 

0 

0.17 

-0.11 

0 

0.43 

0.4 

168 

9.4411 

-0.25 

0 

0.17 

-0.11 

0 

-0.43 

-0.4 

147 

8.8770 

-0.25 

0 

0.17 

-0.11 

0 

-0.43 

0.8 

155 

9.0550 

P-2 


Table  P.3  Correlation  Matrix 


FH 

AP 

DR 

BP 

BRCT 

OST 

DRCT 

REQ 

EBO 

FH 

1 

AP 

0 

1 

DR 

-0.003 

0 

1 

BP 

0.001 

0 

-0.005 

1 

BRCT 

0.003 

0 

-0.015 

0.005 

1 

OST 

0.003 

0 

-0.015 

0.004 

0.015 

1 

DRCT 

-0.003 

0 

0.015 

-0.004 

-0.015 

-0.015 

1 

REQ 

0.499 

0.159 

0.509 

-0.414 

0.407 

0.273 

0.090 

1 

EBO 

0.447 

0.144 

0.481 

-0.657 

-0.217 

0.185 

0.080 

0.736 

1 

P-3 
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